Re: notmuch python bindings corrupt db index (was: gmail importer script)

diff --git a/c3/7f6f08c4b50ca52bf9f8e001422f0a9a38703a b/c3/7f6f08c4b50ca52bf9f8e001422f0a9a38703a
new file mode 100644 (file)
index 0000000..e7596b9
--- /dev/null
+++ b/c3/7f6f08c4b50ca52bf9f8e001422f0a9a38703a
@@ -0,0 +1,98 @@
+Return-Path: <patricktotzke@gmail.com>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+       by olra.theworths.org (Postfix) with ESMTP id ADDF6431FBC\r
+       for <notmuch@notmuchmail.org>; Thu, 13 Dec 2012 08:14:12 -0800 (PST)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -0.799\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-0.799 tagged_above=-999 required=5\r
+       tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1,\r
+       FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+       by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+       with ESMTP id qh+6HCOHU7w1 for <notmuch@notmuchmail.org>;\r
+       Thu, 13 Dec 2012 08:14:12 -0800 (PST)\r
+Received: from mail-we0-f181.google.com (mail-we0-f181.google.com\r
+       [74.125.82.181]) (using TLSv1 with cipher RC4-SHA (128/128 bits))\r
+       (No client certificate requested)\r
+       by olra.theworths.org (Postfix) with ESMTPS id 2D8C2431FB6\r
+       for <notmuch@notmuchmail.org>; Thu, 13 Dec 2012 08:14:12 -0800 (PST)\r
+Received: by mail-we0-f181.google.com with SMTP id t11so942508wey.26\r
+       for <notmuch@notmuchmail.org>; Thu, 13 Dec 2012 08:14:11 -0800 (PST)\r
+DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;\r
+       h=content-type:mime-version:content-transfer-encoding:to:from\r
+       :in-reply-to:cc:references:message-id:user-agent:subject:date;\r
+       bh=GGAFXonHljeBzLhXYRl3l3JnHk7VhSrfmERJy0Mlov0=;\r
+       b=uetKYJUtDiwi6m/0TdgNy4NlDGfs9xDIeY9o38tUUvcXfdNVm1C+Op4Pke7vEazKSs\r
+       Nm0bl1pvhtqo61Y67Qi2dJsiH+jIPE3xC/HkFrrzaxqZJ9m8d8XAsm0edx2fAcH4Itx8\r
+       gI0Qrno8wm4VlSiWDT5gCNY02uX3Qs4veVjaF19sKwP6A8TqYEGl+La8yH13Fey7r4XH\r
+       UkHaVJag/o87kzJ0YQD/tcTxUbFe5WKuASGjzIVqT80SdBKjucFnn+Ve35C+JfuWIDyB\r
+       iu75O+AnKF6A6Ssh92m/9uClRzJWpINfL1fyi+XhwwaG1zIh7vV/uPI6YXqaI5j66r3K\r
+       e5mQ==\r
+Received: by 10.180.80.201 with SMTP id t9mr4359018wix.0.1355415249708;\r
+       Thu, 13 Dec 2012 08:14:09 -0800 (PST)\r
+Received: from localhost (dhcp-91-036.inf.ed.ac.uk. [129.215.91.36])\r
+       by mx.google.com with ESMTPS id o3sm8401416wic.0.2012.12.13.08.14.06\r
+       (version=TLSv1/SSLv3 cipher=OTHER);\r
+       Thu, 13 Dec 2012 08:14:07 -0800 (PST)\r
+Content-Type: text/plain; charset="utf-8"\r
+MIME-Version: 1.0\r
+Content-Transfer-Encoding: quoted-printable\r
+To: "Jason A. Donenfeld" <Jason@zx2c4.com>, Austin Clements <amdragon@mit.edu>\r
+From: Patrick Totzke <patricktotzke@gmail.com>\r
+In-Reply-To:\r
+ <CAHmME9q-NbhRC=Be20aW5Vz+swou419zJxZD4O-3PSTJr+K=iQ@mail.gmail.com>\r
+References:\r
+ <CAHmME9phAD0JKHDf+1KuCPyo4WXpDW5NESAE_iT3Ruv3bEVCuw@mail.gmail.com>\r
+       <20121211182638.27237.98903@brick.lan>\r
+       <CAHmME9qafP5He4hYPEAyrtS4hq3CRNKBQfvxvP=r4L6uDeqt1w@mail.gmail.com>\r
+       <20121212204922.GB6187@mit.edu>\r
+       <CAHmME9q-NbhRC=Be20aW5Vz+swou419zJxZD4O-3PSTJr+K=iQ@mail.gmail.com>\r
+Message-ID: <20121213161600.28349.42261@megatron>\r
+User-Agent: alot/0.3.3+\r
+Subject: Re: notmuch python bindings corrupt db index (was: gmail importer\r
+       script)\r
+Date: Thu, 13 Dec 2012 16:16:00 +0000\r
+Cc: notmuch@notmuchmail.org\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+       <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Thu, 13 Dec 2012 16:14:12 -0000\r
+\r
+Quoting Jason A. Donenfeld (2012-12-13 14:32:53)\r
+> On Wed, Dec 12, 2012 at 9:49 PM, Austin Clements <amdragon@mit.edu> wrote:\r
+> > There should be no way to corrupt the database at this level through\r
+> > the Xapian API, which means nothing libnotmuch can do (much less users\r
+> > of libnotmuch) should be able to corrupt the database.  If you can\r
+> > reproduce the problem, it's probably a serious bug in Xapian, but it\r
+> > could also have been a file system bug or even random file system\r
+> > corruption.\r
+> =\r
+\r
+> Well that's... troubling.\r
+> =\r
+\r
+> Patrick: could you please backup and try to reproduce? Otherwise I'll\r
+> assume this was a one-off situation.\r
+\r
+\r
+I tried it again, concurrently triggered some modifications\r
+and killed the process afer a while.\r
+(my modifications were not flushed during the run of your script as\r
+the index was locked)\r
+The index seems to be OK. So I'm afraid I cannot reproduce this reliably.\r
+/p\r
+\r
+\r