[PATCH v6 00/17] Fix 'notmuch new' atomicity issues

diff --git a/ab/8b20a67906513bf3471d6e146e41eada76e2a9 b/ab/8b20a67906513bf3471d6e146e41eada76e2a9
new file mode 100644 (file)
index 0000000..17f4706
--- /dev/null
+++ b/ab/8b20a67906513bf3471d6e146e41eada76e2a9
@@ -0,0 +1,128 @@
+Return-Path: <amthrax@drake.mit.edu>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+       by olra.theworths.org (Postfix) with ESMTP id 5C0A8431FD0\r
+       for <notmuch@notmuchmail.org>; Sat, 11 Jun 2011 13:04:58 -0700 (PDT)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -0.7\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-0.7 tagged_above=-999 required=5\r
+       tests=[RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+       by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+       with ESMTP id q2pRHr5s6pob for <notmuch@notmuchmail.org>;\r
+       Sat, 11 Jun 2011 13:04:57 -0700 (PDT)\r
+Received: from dmz-mailsec-scanner-6.mit.edu (DMZ-MAILSEC-SCANNER-6.MIT.EDU\r
+       [18.7.68.35])\r
+       by olra.theworths.org (Postfix) with ESMTP id BF8F8431FB6\r
+       for <notmuch@notmuchmail.org>; Sat, 11 Jun 2011 13:04:57 -0700 (PDT)\r
+X-AuditID: 12074423-b7ce8ae000000a29-68-4df3ca633489\r
+Received: from mailhub-auth-4.mit.edu ( [18.7.62.39])\r
+       by dmz-mailsec-scanner-6.mit.edu (Symantec Messaging Gateway) with SMTP\r
+       id D6.55.02601.36AC3FD4; Sat, 11 Jun 2011 16:04:51 -0400 (EDT)\r
+Received: from outgoing.mit.edu (OUTGOING-AUTH.MIT.EDU [18.7.22.103])\r
+       by mailhub-auth-4.mit.edu (8.13.8/8.9.2) with ESMTP id p5BK4udc029922\r
+       for <notmuch@notmuchmail.org>; Sat, 11 Jun 2011 16:04:56 -0400\r
+Received: from drake.mit.edu\r
+       (209-6-116-242.c3-0.arl-ubr1.sbo-arl.ma.cable.rcn.com\r
+       [209.6.116.242]) (authenticated bits=0)\r
+       (User authenticated as amdragon@ATHENA.MIT.EDU)\r
+       by outgoing.mit.edu (8.13.6/8.12.4) with ESMTP id p5BK4tpO005528\r
+       (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NOT)\r
+       for <notmuch@notmuchmail.org>; Sat, 11 Jun 2011 16:04:56 -0400 (EDT)\r
+Received: from amthrax by drake.mit.edu with local (Exim 4.76)\r
+       (envelope-from <amthrax@drake.mit.edu>) id 1QVUQR-0000Gr-1Z\r
+       for notmuch@notmuchmail.org; Sat, 11 Jun 2011 16:04:55 -0400\r
+From: Austin Clements <amdragon@MIT.EDU>\r
+To: notmuch@notmuchmail.org\r
+Subject: [PATCH v6 00/17] Fix 'notmuch new' atomicity issues\r
+Date: Sat, 11 Jun 2011 16:04:26 -0400\r
+Message-Id: <1307822683-848-1-git-send-email-amdragon@mit.edu>\r
+X-Mailer: git-send-email 1.7.5.1\r
+In-Reply-To: <87ei34rnc5.fsf@yoom.home.cworth.org>\r
+References: <87ei34rnc5.fsf@yoom.home.cworth.org>\r
+X-Brightmail-Tracker:\r
+ H4sIAAAAAAAAA+NgFtrBIsWRmVeSWpSXmKPExsUixG6nrpt86rOvwdO51hbXb85kdmD0eLbq\r
+       FnMAYxSXTUpqTmZZapG+XQJXxrWbH5gKGoUqXmx+ztTAeIqvi5GTQ0LARKJ74SZGCFtM4sK9\r
+       9WxdjFwcQgL7GCWa3k5lgnDOMUrMuNLLClIlJPCXSWLLUhUIey+jxNcjpiA2m4CGxLb9y8Em\r
+       iQhIS+y8OxusXljARmLFg99gcRYBVYm/+1rYQGxeATuJex3LWCE2K0hcuTKPBcTmFDCSONG2\r
+       lAlivqFE78EnTBMY+RYwMqxilE3JrdLNTczMKU5N1i1OTszLSy3SNdPLzSzRS00p3cQIDg4X\r
+       5R2Mfw4qHWIU4GBU4uFVXPvZV4g1say4MvcQoyQHk5Ior84JoBBfUn5KZUZicUZ8UWlOavEh\r
+       RgkOZiUR3vXtn3yFeFMSK6tSi/JhUtIcLErivHMl1X2FBNITS1KzU1MLUotgsjIcHEoSvJtP\r
+       Ag0VLEpNT61Iy8wpQUgzcXCCDOcBGn4UpIa3uCAxtzgzHSJ/itGYY+G69YcYOR5v2HSIUYgl\r
+       Lz8vVUqc9z5IqQBIaUZpHtw0WIS/YhQHek6Y9wdIFQ8wOcDNewW0iglolUAp2KqSRISUVAPj\r
+       ydVbb5l8v9HVvFBjp3Lj7Ngr1QvFti0x+Zom0femyaHTRoTdQNmmNlz4r96ruRlvds0vd7Gr\r
+       Y6xaFJ/l8mnyYpngddfSpJ+cSs/WDJjUdDNtmavV3tjvTKx5z4IjBc5nMkd6Gpsdf6PQHPLG\r
+       Xfbv3OXcXT3lLmmMT6Q0NyydkJE17cJJuyVKLMUZiYZazEXFiQAWSbHtywIAAA==\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+       <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Sat, 11 Jun 2011 20:04:58 -0000\r
+\r
+Here's the reworked patch series that uses atomic sections more\r
+heavily rather than changing the removal API.  This is atomic-new-v6\r
+on http://awakening.csail.mit.edu/git/notmuch.git .\r
+\r
+(I was planning to make this series on Monday while stuck on a plane,\r
+but an opportunity presented itself when I needed something better to\r
+do than fix the sink.  As a result, if you can look over this before\r
+Monday, I can use that time to address any further comments.)\r
+\r
+The beginning of the sequence---"test: Fix message when skipping\r
+test_expect_equal* tests" through "new: Defer updating directory\r
+mtimes until the end."---has not changed besides than the minor things\r
+you pointed out earlier.  Here's a quick summary of the differences in\r
+the rest of the sequence:\r
+\r
+lib: Add notmuch_database_{begin,end}_atomic.\r
+  Rebased but otherwise identical.\r
+\r
+lib: Add support for nested atomic sections.\r
+  New.\r
+\r
+lib: Indicate if there are more filenames after removal.\r
+  Rebased but otherwise identical.\r
+\r
+lib: Remove message document directly after removing the last file\r
+name.\r
+  New.  Supersedes the patches that rewrote\r
+  notmuch_database_remove_message and made sync delete messages.\r
+\r
+lib: Add an API to find a message by filename.\r
+  Culled from "lib: Add API's to find by filename and ..." in the old\r
+  series.  What I kept is identical.\r
+\r
+lib: Wrap notmuch_database_add_message in an atomic section.\r
+  New.\r
+\r
+new: Cleanup. Put removed/renamed message count in add_files_state_t.\r
+new: Cleanup. De-duplicate file name removal code.\r
+  Both new, but closely related to previous remove_filename patch.\r
+\r
+new: Synchronize maildir flags eagerly.\r
+  New, but very closely related to previous patch of the same name.\r
+  This patch is simpler now because of the previous two cleanups.\r
+\r
+new: Wrap adding and removing messages in atomic sections.\r
+  Rebased and added an atomic section around removal.\r
+\r
+lib: Improve notmuch_database_{add,remove}_message documentation.\r
+  New.\r
+\r
+Ultimately, I still think the existing removal API is weird because it\r
+forces you to do things like lookup a message by file name just in\r
+case you need that message object after removal tells you that it\r
+didn't actually remove the message.  But I don't know what to do about\r
+it that isn't equally weird.\r
+\r