From: Austin Clements Date: Wed, 23 Apr 2014 21:28:43 +0000 (+2000) Subject: Re: [PATCH] Add configurable changed tag to messages that have been changed on disk X-Git-Url: http://git.tremily.us/?a=commitdiff_plain;h=1ed4dca2bb256546b94ea63ed9460e7b8818567c;p=notmuch-archives.git Re: [PATCH] Add configurable changed tag to messages that have been changed on disk --- diff --git a/a2/cbf9d693a90440acc6537d373ab24d1aa4a67c b/a2/cbf9d693a90440acc6537d373ab24d1aa4a67c new file mode 100644 index 000000000..33471194f --- /dev/null +++ b/a2/cbf9d693a90440acc6537d373ab24d1aa4a67c @@ -0,0 +1,128 @@ +Return-Path: +X-Original-To: notmuch@notmuchmail.org +Delivered-To: notmuch@notmuchmail.org +Received: from localhost (localhost [127.0.0.1]) + by olra.theworths.org (Postfix) with ESMTP id EFBC0431FBF + for ; Wed, 23 Apr 2014 14:28:58 -0700 (PDT) +X-Virus-Scanned: Debian amavisd-new at olra.theworths.org +X-Spam-Flag: NO +X-Spam-Score: -0.7 +X-Spam-Level: +X-Spam-Status: No, score=-0.7 tagged_above=-999 required=5 + tests=[RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled +Received: from olra.theworths.org ([127.0.0.1]) + by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) + with ESMTP id VcMfwEmVbsEp for ; + Wed, 23 Apr 2014 14:28:51 -0700 (PDT) +Received: from dmz-mailsec-scanner-4.mit.edu (dmz-mailsec-scanner-4.mit.edu + [18.9.25.15]) + (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) + (No client certificate requested) + by olra.theworths.org (Postfix) with ESMTPS id 54BEA431FBD + for ; Wed, 23 Apr 2014 14:28:51 -0700 (PDT) +X-AuditID: 1209190f-f790b6d000000c3a-5c-53583092726f +Received: from mailhub-auth-1.mit.edu ( [18.9.21.35]) + (using TLS with cipher AES256-SHA (256/256 bits)) + (Client did not present a certificate) + by dmz-mailsec-scanner-4.mit.edu (Symantec Messaging Gateway) with SMTP + id 7D.67.03130.29038535; Wed, 23 Apr 2014 17:28:50 -0400 (EDT) +Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11]) + by mailhub-auth-1.mit.edu (8.13.8/8.9.2) with ESMTP id s3NLSmnf002716; + Wed, 23 Apr 2014 17:28:49 -0400 +Received: from awakening.csail.mit.edu (awakening.csail.mit.edu [18.26.4.91]) + (authenticated bits=0) + (User authenticated as amdragon@ATHENA.MIT.EDU) + by outgoing.mit.edu (8.13.8/8.12.4) with ESMTP id s3NLSk48019621 + (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); + Wed, 23 Apr 2014 17:28:47 -0400 +Received: from amthrax by awakening.csail.mit.edu with local (Exim 4.80) + (envelope-from ) + id 1Wd4ij-0004yg-2B; Wed, 23 Apr 2014 17:28:45 -0400 +Date: Wed, 23 Apr 2014 17:28:43 -0400 +From: Austin Clements +To: David Mazieres expires 2014-07-05 CEST + +Subject: Re: [PATCH] Add configurable changed tag to messages that have been + changed on disk +Message-ID: <20140423212843.GN25817@mit.edu> +References: <1396800683-9164-1-git-send-email-eg@gaute.vetsj.com> + <87wqf2gqig.fsf@ta.scs.stanford.edu> +MIME-Version: 1.0 +Content-Type: text/plain; charset=us-ascii +Content-Disposition: inline +In-Reply-To: <87wqf2gqig.fsf@ta.scs.stanford.edu> +User-Agent: Mutt/1.5.21 (2010-09-15) +X-Brightmail-Tracker: + H4sIAAAAAAAAA+NgFuplleLIzCtJLcpLzFFi42IR4hRV1p1kEBFssOA1v0XT50usFqsWxVtc + vzmT2YHZ48e/ZjaPZ6tuMXtc+ruNKYA5issmJTUnsyy1SN8ugSvj57bTLAWzhSounj7K0sB4 + hK+LkZNDQsBEYvLFD0wQtpjEhXvr2boYuTiEBGYzSdyctwbK2cgo0X96IjOEc5pJYtb1lawQ + zhJGicPnLoH1swioSky7t5gRxGYT0JDYtn85mC0iUCwx8fMFZhCbWcBGYsLH12wgtrBAnMS0 + yzvAenkFdCSevl4AViMkkCFxYsYhFoi4oMTJmU9YIHq1JG78ewlUzwFkS0ss/8cBEuYUMJR4 + tLqdFcQWFVCRmHJyG9sERqFZSLpnIemehdC9gJF5FaNsSm6Vbm5iZk5xarJucXJiXl5qka6J + Xm5miV5qSukmRnCoS/LvYPx2UOkQowAHoxIPr8Tl8GAh1sSy4srcQ4ySHExKorzJmhHBQnxJ + +SmVGYnFGfFFpTmpxYcYJTiYlUR4N2sD5XhTEiurUovyYVLSHCxK4rxvra2ChQTSE0tSs1NT + C1KLYLIyHBxKErxL9YEaBYtS01Mr0jJzShDSTBycIMN5gIbPB6nhLS5IzC3OTIfIn2JUlBLn + vagHlBAASWSU5sH1wlLRK0ZxoFeEeZtA2nmAaQyu+xXQYCagwQUTwkEGlyQipKQaGAXir9hG + mq/sWHJoN3vHPBMG4+NMZqKyNVtn7Pb3fdxQb2hbHXHgTeGR61L3hIr/XJk6W5nx+e7zL1nN + dt5cFXnfbVpfAqPp36g/6wriPmxhF/cw9jqjcii6crKwUbga+8pWHyPDFE2f5E8Rrdde/9h5 + IPn+1dD9z14Zz+XQ6TlzlPXaGxG7v0osxRmJhlrMRcWJAFmmB+ggAwAA +Cc: notmuch@notmuchmail.org +X-BeenThere: notmuch@notmuchmail.org +X-Mailman-Version: 2.1.13 +Precedence: list +List-Id: "Use and development of the notmuch mail system." + +List-Unsubscribe: , + +List-Archive: +List-Post: +List-Help: +List-Subscribe: , + +X-List-Received-Date: Wed, 23 Apr 2014 21:28:59 -0000 + +Quoth David Mazieres on Apr 06 at 10:19 pm: +> Gaute Hope writes: +> +> > When one of the source files for a message is changed on disk, renamed, +> > deleted or a new source file is added. A configurable changed tag is +> > is added. The tag can be configured under the option 'changed_tags' in +> > the [new] section, the default is none. Tests have been updated to +> > accept the new config option. +> > +> > notmuch-setup now asks for a changed tag after the new tags question. +> > +> > This could be useful for for example 'afew' to detect remote changes in +> > IMAP folders and update the FolderNameFilter to also add tags or remove +> > tags when a _existing_ message has been added to or removed from a +> > maildir. +> +> I think this is the wrong way to achieve such functionality, because +> then the change tag A) is expensive to remove, B) is easy to misuse +> (remember to call fsync everywhere before deleting the change tag), and +> C) can be used by only one application. +> +> A better approach would be to add a new "modtime" xapian value that is +> updated whenever the tags or any other terms (such as XFDIRENTRY) are +> added to or deleted from a docid. If it's a Xapian value, rather than a +> term, then modtime will be queriable just like date, allowing multiple +> applications to query all docids modified since the last time they ran. + +I'd like to have efficient change detection, too. In my case, I'd +like to use it to support efficient live search and show updates. The +design I'd sketched out for that used a log rather than ctimes, and +I'm curious if you have thoughts on the relative merits and +suitability for tag sync of these approaches. + +I'd been leaning toward logging because it can capture changes to +things that aren't represented as documents in the database, such as +thread membership. This probably doesn't matter for synchronization, +but it makes it much easier to figure out which threads in thread +search results need to be refreshed. A log can also capture message +deletion easily, while ctimes would require tombstones (which may be a +good idea for other reasons [1]). + +On the other hand, logging is obviously more mechanism than ctimes, +and probably requires some garbage collection story. + +[1] id:20140421162058.GE25817@mit.edu