--- /dev/null
+Return-Path: <amdragon@mit.edu>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+ by olra.theworths.org (Postfix) with ESMTP id 5AD0B431FD0\r
+ for <notmuch@notmuchmail.org>; Tue, 20 Dec 2011 07:03:57 -0800 (PST)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -0.7\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-0.7 tagged_above=-999 required=5\r
+ tests=[RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+ by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+ with ESMTP id r-2MZINkMl77 for <notmuch@notmuchmail.org>;\r
+ Tue, 20 Dec 2011 07:03:56 -0800 (PST)\r
+Received: from dmz-mailsec-scanner-5.mit.edu (DMZ-MAILSEC-SCANNER-5.MIT.EDU\r
+ [18.7.68.34])\r
+ by olra.theworths.org (Postfix) with ESMTP id 99C8A431FB6\r
+ for <notmuch@notmuchmail.org>; Tue, 20 Dec 2011 07:03:56 -0800 (PST)\r
+X-AuditID: 12074422-b7fd66d0000008f9-fd-4ef0a3db0c27\r
+Received: from mailhub-auth-2.mit.edu ( [18.7.62.36])\r
+ by dmz-mailsec-scanner-5.mit.edu (Symantec Messaging Gateway) with SMTP\r
+ id C8.D7.02297.BD3A0FE4; Tue, 20 Dec 2011 10:03:56 -0500 (EST)\r
+Received: from outgoing.mit.edu (OUTGOING-AUTH.MIT.EDU [18.7.22.103])\r
+ by mailhub-auth-2.mit.edu (8.13.8/8.9.2) with ESMTP id pBKF3swd018335; \r
+ Tue, 20 Dec 2011 10:03:55 -0500\r
+Received: from awakening.csail.mit.edu (awakening.csail.mit.edu [18.26.4.91])\r
+ (authenticated bits=0)\r
+ (User authenticated as amdragon@ATHENA.MIT.EDU)\r
+ by outgoing.mit.edu (8.13.6/8.12.4) with ESMTP id pBKF3q2n025101\r
+ (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NOT);\r
+ Tue, 20 Dec 2011 10:03:53 -0500 (EST)\r
+Received: from amthrax by awakening.csail.mit.edu with local (Exim 4.77)\r
+ (envelope-from <amdragon@mit.edu>)\r
+ id 1Rd1FX-0004bN-Vv; Tue, 20 Dec 2011 10:05:04 -0500\r
+Date: Tue, 20 Dec 2011 10:05:03 -0500\r
+From: Austin Clements <amdragon@MIT.EDU>\r
+To: David Edmondson <dme@dme.org>\r
+Subject: Re: [PATCH 0/5] Store message modification times in the DB\r
+Message-ID: <20111220150503.GE10376@mit.edu>\r
+References: <1323796305-28789-1-git-send-email-schnouki@schnouki.net>\r
+ <cunk45szfiu.fsf@hotblack-desiato.hh.sledj.net>\r
+ <20111219194821.GA10376@mit.edu>\r
+ <cun8vm7zlqv.fsf@hotblack-desiato.hh.sledj.net>\r
+MIME-Version: 1.0\r
+Content-Type: text/plain; charset=us-ascii\r
+Content-Disposition: inline\r
+In-Reply-To: <cun8vm7zlqv.fsf@hotblack-desiato.hh.sledj.net>\r
+User-Agent: Mutt/1.5.21 (2010-09-15)\r
+X-Brightmail-Tracker:\r
+ H4sIAAAAAAAAA+NgFuphleLIzCtJLcpLzFFi42IRYrdT0b2z+IOfwYqdTBb77mxhsrh+cyaz\r
+ xb5+fwdmj13P/zJ5PFt1i9ljyqy57AHMUVw2Kak5mWWpRfp2CVwZ8ye9ZCr4oVjx7j97A+N1\r
+ qS5GTg4JAROJy5uXM0LYYhIX7q1n62Lk4hAS2McoMX/5T3YIZwOjxIM7O5khnJNMEoduLWIH\r
+ aRESWMIosXJHEYjNIqAq0TDjBROIzSagIbFtP8RYEQFFif/fVoDVMws4SKw82QtmCws4S2z+\r
+ uZMVxOYV0JHovPKaCWLBGUaJOds/M0MkBCVOznzCAtGsJXHj30ugIg4gW1pi+T8OkDCngI3E\r
+ yqeTwfaKCqhITDm5jW0Co9AsJN2zkHTPQuhewMi8ilE2JbdKNzcxM6c4NVm3ODkxLy+1SNdU\r
+ LzezRC81pXQTIzjQXZR2MP48qHSIUYCDUYmHd2XTez8h1sSy4srcQ4ySHExKorwnF37wE+JL\r
+ yk+pzEgszogvKs1JLT7EKMHBrCTCe6wFKMebklhZlVqUD5OS5mBREudV13rnJySQnliSmp2a\r
+ WpBaBJOV4eBQkuCVAUa0kGBRanpqRVpmTglCmomDE2Q4D9Bwf5Aa3uKCxNzizHSI/ClGRSlx\r
+ XieQhABIIqM0D64XloheMYoDvSLM6wNSxQNMYnDdr4AGMwEN3uYMNrgkESEl1cDIO79h5fP3\r
+ O+/9lcuTZJVq71GfNr/PuqBswV+HM6qX7rSfa1zvdvx8wqmn/at+r1nNcT3Ny2zWW8/25fOW\r
+ OxpIb3iVI2r6TqdToPBRhWuKa9CDSyWK/YvFNk768vz6XBfO7+1zzi/caaY3h3Gny7kH0kdr\r
+ jYJ7drK1tBzY9eod327RLwudz/gkKrEUZyQaajEXFScCAH+5yf8fAwAA\r
+Cc: notmuch@notmuchmail.org\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+ <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Tue, 20 Dec 2011 15:03:57 -0000\r
+\r
+Quoth David Edmondson on Dec 20 at 8:32 am:\r
+> > == Two-way "merge" from host R to host L ==\r
+> > \r
+> > Per-host state:\r
+> > - last_mtime: Map from remote hosts to last sync mtime\r
+> \r
+> With the proposed changes it seems that the state required on each host\r
+> would live within the Xapian database (to be extracted with 'dump').\r
+\r
+It certainly could. I haven't thought about how any of this would\r
+integrate with dump, or if it necessarily should. A related question\r
+is how bootstrap should work. For example, if you add another host,\r
+what's the best way to bring it up to speed without, say, overwriting\r
+your tags everywhere with your initial tags? In general, when a new\r
+message arrives, how do you get the hosts to agree on its tags and\r
+what happens if one host tags it before another host sees it?\r
+\r
+> > new_mtime = last_mtime[R]\r
+> > For msgid, mtime, tags in messages on host R with mtime >= last_mtime[R]:\r
+> > If mtime > local mtime of msgid:\r
+> > Set local tags of msgid to tags\r
+> > new_mtime = max(new_mtime, mtime)\r
+> > last_mtime[R] = new_mtime\r
+> > \r
+> > This has the advantage of keeping very little state, but the\r
+> > synchronization is also quite primitive. If two hosts change a\r
+> > message's tags in different ways between synchronizations, the more\r
+> > recent of the two will override the full set of tags on that message.\r
+> > This does not strictly require tombstones, though if you make a tag\r
+> > change and then delete the message before a sync, the tag change will\r
+> > be lost without some record of that state.\r
+> \r
+> Does this matter? If the tag on a deleted message is changed, does\r
+> anyone care?\r
+\r
+That depends on what sort of synchronization model you're expecting.\r
+If you're expecting git-style synchronization where all that matters\r
+is the state and not the order things happened in, then this is\r
+exactly what you'd expect. If you're expecting something more nuanced\r
+that knows about the order you did things in across hosts between\r
+synchronizations (which I think can only lead to more unintuitive\r
+corner-cases, but some people seem to expect), then this could be\r
+surprising.\r
+\r
+> > Also, this obviously depends heavily on synchronized clocks.\r
+> > \r
+> > \r
+> > == Three-way merge from host R to host L ==\r
+> > \r
+> > Per-host state:\r
+> > - last_mtime: Map from remote hosts to last sync mtime\r
+> > - last_sync: Map from remote hosts to the tag database as of the last sync\r
+> \r
+> Any ideas where this state might be kept?\r
+\r
+It could also be stored in Xapian (in user keys or as additional\r
+message metadata). That would certainly be simplest and would avoid\r
+hairy atomicity issues. OTOH, it's not the end of the world if\r
+last_sync doesn't get updated atomically, especially if we can at\r
+least guarantee last_sync is fully updated and on disk before we\r
+update last_mtime.\r
+\r
+> > new_mtime = last_mtime[R]\r
+> > for msgid, mtime, r_tags in messages on host R with mtime >= last_mtime[R]:\r
+> > my_tags = local tags of msgid\r
+> > last_tags = last_sync[R][msgid]\r
+> > for each tag that differs between my_tags and r_tags:\r
+> > if tag is in last_tags: remove tag locally\r
+> > else: add tag locally\r
+> > last_sync[R][msgid] = tags\r
+> > new_mtime = max(new_mtime, mtime)\r
+> > Delete stale messages from last_sync[R] (using tombstones or something)\r
+> > last_mtime[R] = new_mtime\r
+> > \r
+> > This protocol requires significantly more state, but can also\r
+> > reconstruct per-tag changes. Conflict resolution is equivalent to\r
+> > what git would do and is based solely on the current local and remote\r
+> > state and the common ancestor state. This can lead to unintuitive\r
+> > results if a tag on a message has gone through multiple changes on\r
+> > both hosts since the last sync (though, I argue, there are no\r
+> > intuitive results in such situations). Tombstones are only required\r
+> > to garbage collect sync state (and other techniques could be used for\r
+> > that). This also does not depend on time synchronization (though,\r
+> > like any mtime solution, it does depend on mtime monotonicity). The\r
+> > algorithm would work equally well with sequence numbers.\r
+> > \r
+> > I tried coming up with a third algorithm that used mtimes to resolve\r
+> > tagging conflicts, but without per-tag mtimes it degenerated into the\r
+> > first algorithm.\r
+> \r
+> dme.\r
projects / notmuch-archives.git / commitdiff