--- /dev/null
+Return-Path: <eg@gaute.vetsj.com>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+ by olra.theworths.org (Postfix) with ESMTP id 13A14431FBC\r
+ for <notmuch@notmuchmail.org>; Thu, 10 Apr 2014 14:11:39 -0700 (PDT)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -0.7\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-0.7 tagged_above=-999 required=5\r
+ tests=[RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+ by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+ with ESMTP id 2cZf5c3w9GcH for <notmuch@notmuchmail.org>;\r
+ Thu, 10 Apr 2014 14:11:35 -0700 (PDT)\r
+Received: from mail-lb0-f171.google.com (mail-lb0-f171.google.com\r
+ [209.85.217.171]) (using TLSv1 with cipher RC4-SHA (128/128 bits))\r
+ (No client certificate requested)\r
+ by olra.theworths.org (Postfix) with ESMTPS id 1FA18431FAE\r
+ for <notmuch@notmuchmail.org>; Thu, 10 Apr 2014 14:11:34 -0700 (PDT)\r
+Received: by mail-lb0-f171.google.com with SMTP id w7so2777282lbi.2\r
+ for <notmuch@notmuchmail.org>; Thu, 10 Apr 2014 14:11:32 -0700 (PDT)\r
+X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;\r
+ d=1e100.net; s=20130820;\r
+ h=x-gm-message-state:content-type:from:to:cc:subject:in-reply-to\r
+ :references:date:message-id:user-agent:content-transfer-encoding;\r
+ bh=3SgpQsUytmES8TLpkjCznp8F6qKnc2NslVrnPnrHIfY=;\r
+ b=VdiM+DrKehrOxZSCMHXI2hnAuYeNJPTPH6Fj3VxPlKlkur3VqVPaZFW9HqFF6UUJjN\r
+ d+r2nOsEmVLXyfdqfLe6EtVvrR3VWL/sT8euE1np3rx9xMbJ+swkIdzskZDg8+6nGm/h\r
+ 1U5zzc+l3SQENbylgAAERctm1hVCPmnOlYhLqC2qz+BZi8pnMdL9JopULnVRtTeM8A3H\r
+ U+gnbUjZo5HDzIkAtp+WP/Z5ZnPxsVvPlwjZ5/x5PO3KOfMWSTyUZA7wn+SCpzTpdaDp\r
+ yANn7UX+JbjiM5XCaujHIXbOZEslseVY9KY4fYBdZby94vZkIooh9EyjsW9k41tq7fWU\r
+ KT3w==\r
+X-Gm-Message-State:\r
+ ALoCoQmvyobhsQ+Ed0VSyHO5124h52ANmYny6lFbAz4EJibHE2a0j/yz1OFsaV4NLZfj5qc8gxIV\r
+X-Received: by 10.152.42.164 with SMTP id p4mr13705398lal.5.1397164292100;\r
+ Thu, 10 Apr 2014 14:11:32 -0700 (PDT)\r
+Received: from localhost (cD572BF51.dhcp.as2116.net. [81.191.114.213])\r
+ by mx.google.com with ESMTPSA id g8sm5197206laf.0.2014.04.10.14.11.29\r
+ for <multiple recipients>\r
+ (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);\r
+ Thu, 10 Apr 2014 14:11:30 -0700 (PDT)\r
+Content-Type: text/plain; charset=UTF-8\r
+From: Gaute Hope <eg@gaute.vetsj.com>\r
+To: David Mazieres expires 2014-07-09 PDT\r
+ <mazieres-gzp7rfravpipmg73ew8cs6n6t6@temporary-address.scs.stanford.edu>\r
+Subject: Re: [PATCH] Add configurable changed tag to messages that have been\r
+ changed on disk\r
+In-reply-to: <87wqexnqvb.fsf@ta.scs.stanford.edu>\r
+References: <1396800683-9164-1-git-send-email-eg@gaute.vetsj.com>\r
+ <87wqf2gqig.fsf@ta.scs.stanford.edu> <1397140962-sup-6514@qwerzila>\r
+ <87wqexnqvb.fsf@ta.scs.stanford.edu>\r
+Date: Thu, 10 Apr 2014 23:10:20 +0200\r
+Message-Id: <1397163239-sup-5101@qwerzila>\r
+User-Agent: Sup/git\r
+Content-Transfer-Encoding: 8bit\r
+Cc: notmuch <notmuch@notmuchmail.org>\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+ <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Thu, 10 Apr 2014 21:11:39 -0000\r
+\r
+Excerpts from dm-list-email-notmuch's message of 2014-04-10 17:31:04 +0200:\r
+> Gaute Hope <eg@gaute.vetsj.com> writes:\r
+>\r
+> >> A better approach would be to add a new "modtime" xapian value that is\r
+> >> updated whenever the tags or any other terms (such as XFDIRENTRY) are\r
+> >> added to or deleted from a docid. If it's a Xapian value, rather than a\r
+> >> term, then modtime will be queriable just like date, allowing multiple\r
+> >> applications to query all docids modified since the last time they ran.\r
+> >>\r
+> >> [... snip]\r
+> >\r
+> > This could also solve it, and probably have more uses. I don't quite see\r
+> > how the opposite problem (for my use case) can be solved by this without\r
+> > using a 'localchange' tag. This is to sync tag to maildir sync, when a\r
+> > new tag has been added (by e.g. a user interaction in a client) it needs\r
+> > to be copied to the maildir, if it is not done in the same go a\r
+> > different application won't know whether the change was local or remote.\r
+> > How did you solve this?\r
+>\r
+> Why don't you just set maildir.synchronize_flags=true? When I\r
+> synchronize mail across machines, I start by concurrently running\r
+> "notmuch new" on both the local and remote machines, which picks up all\r
+> the changed maildir flags. Then I synchronize the mail and the tags\r
+> between the two maildirs. If maildir.synchronize=true, then atomically\r
+> with setting the new tags I call notmuch_message_tags_to_maildir_flags()\r
+> to sync the new tags to the maildir.\r
+\r
+I am talking about syncing tags to a maildir _folder_, not flags. It\r
+could be implemented as maildir.synchronize is now, but it would be a\r
+larger feature which could work in a lot of different ways.\r
+\r
+> The maildir flags question seems kind of independent of what we are\r
+> talking about, which is just having an incremental way of examining the\r
+> database. Right now, I have to scan everything to find tags that have\r
+> changed since the last synchronization event. If I had modtime (or\r
+> really it should be called "ctime", like inode change time), then I\r
+> could look at only the few messages that changed, and it would probably\r
+> shave 250msec off polling new mail for a 100,000-message maildir.\r
+>\r
+> Note you can't use the file system ctime/mtime because the file system\r
+> may have changed since the last time you ran notmuch new.\r
+\r
+If you have a unreliable clock or use a badly configured system you\r
+could risk detecting changes in the case where application time stamp is\r
+set in the future, a mod time now. Then the app won't know there has\r
+been a change. The same could happen if the clock is in the past, and\r
+the modtime is set, the clock is updated and the app won't know there\r
+has been a change.\r
+\r
+The only way to know is to do a full scan of the entire db. This could\r
+be very expansive, and comparable to initial indexing, for some actions.\r
+\r
+You would not necessarily, or reliably, be able to detect this.\r
+\r
+With an internal tick this wouldn't be an issue.\r
+\r
+> > I would suggest using a Xapian- or Index-time which gets a tick\r
+> > everytime a modification is made to the index.\r
+>\r
+> Exactly. It could be a tick, or just the current time of day if your\r
+> clock does not go backwards. (I'd be willing to do a full scan if the\r
+> clock ever goes backwards.) The advantage of time is that you don't\r
+> have to synchronously update some counter.\r
+>\r
+> > Atomic operations could operate on the same time in case this\r
+> > distinction turns out to be useful. Perhaps something like this\r
+> > already exists in Xapian?\r
+>\r
+> I don't think it's important for atomic operations to have the same\r
+> timestamp. All that's important is that you be able to diff the\r
+> database between the last time you scanned it.\r
+\r
+Yeah, it is not necessary for anything I am planning on doing, but it\r
+would be a way for other apps to know that a set of changes were done at\r
+the same time.\r
+\r
+> > This way clock skew, clock resolution (lots of operations happening in\r
+> > the same second, msec or nanosec) problems won't be an issue. The crux\r
+> > will be to make sure all write-operations trigger a tick on the\r
+> > indextime.\r
+>\r
+> Clock skew is not really an issue. It takes years to amass hundreds of\r
+> thousands of email messages. So adding 5 minutes of slop is not a big\r
+> deal--you'll just scan a few messages needlessly.\r
+\r
+Yes, but you risk missing changes without knowing. That is an issue for\r
+my use case.\r
+\r
+\r
+> Making sure the write-operations update the time should be easy. Most\r
+> or all of the changes are probably funneled through\r
+> _notmuch_message_sync. Worst case, there are only 9 places in the\r
+> source code that make use of a Xapian:WritableDatabase, so I'm pretty\r
+> confident total changes wouldn't be much more than 50 lines of code.\r
+\r
+Yes :)\r
+\r
+> I would do it myself if there were any kind of indication that such a\r
+> change could be upstreamed. I brought this up in January, 2011, and\r
+> didn't get a huge amount of interest in the ctime idea. But I was also\r
+> a lot less focused on what I needed. Now that I have a working\r
+> distributed setup and am actually using notmuch for my mail, I have a\r
+> much better understanding of what is needed.\r
+\r
+Would be great if it could be included.. I guess a comment from\r
+one/some of the notmuch-gurus could clarify?\r
+\r
+\r
+- gaute\r
projects / notmuch-archives.git / commitdiff