--- /dev/null
+Return-Path:\r
+ <return-hrv8eihpgeb84a37siqwh9vg2i@temporary-address.scs.stanford.edu>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+ by olra.theworths.org (Postfix) with ESMTP id 2ADF3431FBD\r
+ for <notmuch@notmuchmail.org>; Thu, 10 Apr 2014 08:32:28 -0700 (PDT)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -2.3\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-2.3 tagged_above=-999 required=5\r
+ tests=[RCVD_IN_DNSWL_MED=-2.3] autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+ by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+ with ESMTP id KbDkTKtL8psJ for <notmuch@notmuchmail.org>;\r
+ Thu, 10 Apr 2014 08:32:21 -0700 (PDT)\r
+Received: from market.scs.stanford.edu (market.scs.stanford.edu [171.66.3.10])\r
+ (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))\r
+ (No client certificate requested)\r
+ by olra.theworths.org (Postfix) with ESMTPS id 136E8431FBC\r
+ for <notmuch@notmuchmail.org>; Thu, 10 Apr 2014 08:32:20 -0700 (PDT)\r
+Received: from market.scs.stanford.edu (localhost.scs.stanford.edu\r
+ [127.0.0.1]) by market.scs.stanford.edu (8.14.7/8.14.7) with ESMTP id\r
+ s3AFV4Ia005872; Thu, 10 Apr 2014 08:31:04 -0700 (PDT)\r
+Received: (from dm@localhost)\r
+ by market.scs.stanford.edu (8.14.7/8.14.7/Submit) id s3AFV49m008022;\r
+ Thu, 10 Apr 2014 08:31:04 -0700 (PDT)\r
+X-Authentication-Warning: market.scs.stanford.edu: dm set sender to\r
+ return-hrv8eihpgeb84a37siqwh9vg2i@ta.scs.stanford.edu using -f\r
+From: dm-list-email-notmuch@scs.stanford.edu\r
+To: Gaute Hope <eg@gaute.vetsj.com>\r
+Subject: Re: [PATCH] Add configurable changed tag to messages that have been\r
+ changed on disk\r
+In-Reply-To: <1397140962-sup-6514@qwerzila>\r
+References: <1396800683-9164-1-git-send-email-eg@gaute.vetsj.com>\r
+ <87wqf2gqig.fsf@ta.scs.stanford.edu> <1397140962-sup-6514@qwerzila>\r
+Date: Thu, 10 Apr 2014 08:31:04 -0700\r
+Message-ID: <87wqexnqvb.fsf@ta.scs.stanford.edu>\r
+MIME-Version: 1.0\r
+Content-Type: text/plain\r
+Cc: notmuch <notmuch@notmuchmail.org>\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+Reply-To: David Mazieres expires 2014-07-09 PDT\r
+ <mazieres-gzp7rfravpipmg73ew8cs6n6t6@temporary-address.scs.stanford.edu>\r
+List-Id: "Use and development of the notmuch mail system."\r
+ <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Thu, 10 Apr 2014 15:32:28 -0000\r
+\r
+Gaute Hope <eg@gaute.vetsj.com> writes:\r
+\r
+>> A better approach would be to add a new "modtime" xapian value that is\r
+>> updated whenever the tags or any other terms (such as XFDIRENTRY) are\r
+>> added to or deleted from a docid. If it's a Xapian value, rather than a\r
+>> term, then modtime will be queriable just like date, allowing multiple\r
+>> applications to query all docids modified since the last time they ran.\r
+>>\r
+>> [... snip]\r
+>\r
+> This could also solve it, and probably have more uses. I don't quite see\r
+> how the opposite problem (for my use case) can be solved by this without\r
+> using a 'localchange' tag. This is to sync tag to maildir sync, when a\r
+> new tag has been added (by e.g. a user interaction in a client) it needs\r
+> to be copied to the maildir, if it is not done in the same go a\r
+> different application won't know whether the change was local or remote.\r
+> How did you solve this?\r
+\r
+Why don't you just set maildir.synchronize_flags=true? When I\r
+synchronize mail across machines, I start by concurrently running\r
+"notmuch new" on both the local and remote machines, which picks up all\r
+the changed maildir flags. Then I synchronize the mail and the tags\r
+between the two maildirs. If maildir.synchronize=true, then atomically\r
+with setting the new tags I call notmuch_message_tags_to_maildir_flags()\r
+to sync the new tags to the maildir.\r
+\r
+The maildir flags question seems kind of independent of what we are\r
+talking about, which is just having an incremental way of examining the\r
+database. Right now, I have to scan everything to find tags that have\r
+changed since the last synchronization event. If I had modtime (or\r
+really it should be called "ctime", like inode change time), then I\r
+could look at only the few messages that changed, and it would probably\r
+shave 250msec off polling new mail for a 100,000-message maildir.\r
+\r
+Note you can't use the file system ctime/mtime because the file system\r
+may have changed since the last time you ran notmuch new.\r
+\r
+> I would suggest using a Xapian- or Index-time which gets a tick\r
+> everytime a modification is made to the index.\r
+\r
+Exactly. It could be a tick, or just the current time of day if your\r
+clock does not go backwards. (I'd be willing to do a full scan if the\r
+clock ever goes backwards.) The advantage of time is that you don't\r
+have to synchronously update some counter.\r
+\r
+> Atomic operations could operate on the same time in case this\r
+> distinction turns out to be useful. Perhaps something like this\r
+> already exists in Xapian?\r
+\r
+I don't think it's important for atomic operations to have the same\r
+timestamp. All that's important is that you be able to diff the\r
+database between the last time you scanned it.\r
+\r
+> This way clock skew, clock resolution (lots of operations happening in\r
+> the same second, msec or nanosec) problems won't be an issue. The crux\r
+> will be to make sure all write-operations trigger a tick on the\r
+> indextime.\r
+\r
+Clock skew is not really an issue. It takes years to amass hundreds of\r
+thousands of email messages. So adding 5 minutes of slop is not a big\r
+deal--you'll just scan a few messages needlessly.\r
+\r
+Making sure the write-operations update the time should be easy. Most\r
+or all of the changes are probably funneled through\r
+_notmuch_message_sync. Worst case, there are only 9 places in the\r
+source code that make use of a Xapian:WritableDatabase, so I'm pretty\r
+confident total changes wouldn't be much more than 50 lines of code.\r
+\r
+I would do it myself if there were any kind of indication that such a\r
+change could be upstreamed. I brought this up in January, 2011, and\r
+didn't get a huge amount of interest in the ctime idea. But I was also\r
+a lot less focused on what I needed. Now that I have a working\r
+distributed setup and am actually using notmuch for my mail, I have a\r
+much better understanding of what is needed.\r
+\r
+David\r
projects / notmuch-archives.git / commitdiff