From 8d3bccca714337531f826f9f2f156bb24c863c0a Mon Sep 17 00:00:00 2001 From: David Mazieres Date: Thu, 24 Apr 2014 02:00:10 +1700 Subject: [PATCH] Re: [PATCH] Add configurable changed tag to messages that have been changed on disk --- c7/8fc64abd567c69ca0fff1e6ce3031b01fcb89c | 93 +++++++++++++++++++++++ 1 file changed, 93 insertions(+) create mode 100644 c7/8fc64abd567c69ca0fff1e6ce3031b01fcb89c diff --git a/c7/8fc64abd567c69ca0fff1e6ce3031b01fcb89c b/c7/8fc64abd567c69ca0fff1e6ce3031b01fcb89c new file mode 100644 index 000000000..e52901953 --- /dev/null +++ b/c7/8fc64abd567c69ca0fff1e6ce3031b01fcb89c @@ -0,0 +1,93 @@ +Return-Path: + +X-Original-To: notmuch@notmuchmail.org +Delivered-To: notmuch@notmuchmail.org +Received: from localhost (localhost [127.0.0.1]) + by olra.theworths.org (Postfix) with ESMTP id 53FF6431FBF + for ; Wed, 23 Apr 2014 02:00:20 -0700 (PDT) +X-Virus-Scanned: Debian amavisd-new at olra.theworths.org +X-Spam-Flag: NO +X-Spam-Score: -2.3 +X-Spam-Level: +X-Spam-Status: No, score=-2.3 tagged_above=-999 required=5 + tests=[RCVD_IN_DNSWL_MED=-2.3] autolearn=disabled +Received: from olra.theworths.org ([127.0.0.1]) + by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) + with ESMTP id OI-Q3H0aTVKv for ; + Wed, 23 Apr 2014 02:00:15 -0700 (PDT) +Received: from market.scs.stanford.edu (market.scs.stanford.edu [171.66.3.10]) + (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) + (No client certificate requested) + by olra.theworths.org (Postfix) with ESMTPS id B53CF431FBD + for ; Wed, 23 Apr 2014 02:00:15 -0700 (PDT) +Received: from market.scs.stanford.edu (localhost.scs.stanford.edu + [127.0.0.1]) by market.scs.stanford.edu (8.14.7/8.14.7) with ESMTP id + s3N90B63021064; Wed, 23 Apr 2014 02:00:11 -0700 (PDT) +Received: (from dm@localhost) + by market.scs.stanford.edu (8.14.7/8.14.7/Submit) id s3N90BRf001972; + Wed, 23 Apr 2014 02:00:11 -0700 (PDT) +X-Authentication-Warning: market.scs.stanford.edu: dm set sender to + return-z6xc2xnzh5j88kwfw4mkg79bw2@ta.scs.stanford.edu using -f +From: David Mazieres +To: Gaute Hope , David Bremner , + notmuch +Subject: Re: [PATCH] Add configurable changed tag to messages that have been + changed on disk +In-Reply-To: <1398237865-sup-624@qwerzila> +References: <1396800683-9164-1-git-send-email-eg@gaute.vetsj.com> + <87wqf2gqig.fsf@ta.scs.stanford.edu> <1397140962-sup-6514@qwerzila> + <87wqexnqvb.fsf@ta.scs.stanford.edu> <1397163239-sup-5101@qwerzila> + <87d2g9ja0h.fsf@maritornes.cs.unb.ca> <1398237865-sup-624@qwerzila> +Date: Wed, 23 Apr 2014 02:00:10 -0700 +Message-ID: <87ioq0l8th.fsf@ta.scs.stanford.edu> +MIME-Version: 1.0 +Content-Type: text/plain +X-BeenThere: notmuch@notmuchmail.org +X-Mailman-Version: 2.1.13 +Precedence: list +Reply-To: David Mazieres expires 2014-07-22 PDT + +List-Id: "Use and development of the notmuch mail system." + +List-Unsubscribe: , + +List-Archive: +List-Post: +List-Help: +List-Subscribe: , + +X-List-Received-Date: Wed, 23 Apr 2014 09:00:20 -0000 + +Gaute Hope writes: + +> A db-tick or a _good_ ctime solution can as far as I can see solve both +> David M's (correct me if I am wrong) and my purposes, as well as +> probably have more use cases in the future. It would even be an +> interesting direct search: show me everything that changed lately, +> sorted. + +I could live with a db-tick scheme. I would prefer a ctime scheme, +since then I can answer questions such as "what has changed in the last +five minutes"? I mean all kinds of other stuff starts to break if your +clock goes backwards on a mail server machine, not the least of which is +that incremental backups will fail silently, so you risk losing your +mail. + +A middle ground might be to use the maximum of two values: 1) the +time-of-day at which notmuch started executing, and 2) the highest ctime +in the database plus 100 microseconds (leaving plenty of slop to store +timestamps as IEEE doubles with 52 significant bits). Since the values +will be Btree-indexed, computing the max plus one will be cheap. + +Incidentally, if you are really this paranoid about time stamps, it +should bother you that notmuch's directory timestamps only have one +second granularity. It's not that hard to get a new message delivered +in the same second that notmuch new finished running. In my +synchronizer, I convert st_mtim (a struct timespec) into a double and +keep that plus size in the database to decide if I need to re-hash +files. But for directories, I'm stuck with NOTMUCH_VALUE_TIMESTAMP, +which are quantized to the second. (Ironically, I think +Xapian::sortable_serialize converts time_ts to doubles anyway, so +avoiding st_mtim is not really helping performance.) + +David -- 2.26.2