--- /dev/null
+Return-Path: <amdragon@mit.edu>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+ by olra.theworths.org (Postfix) with ESMTP id 23D91431FBD\r
+ for <notmuch@notmuchmail.org>; Fri, 27 Jun 2014 12:37:12 -0700 (PDT)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -0.7\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-0.7 tagged_above=-999 required=5\r
+ tests=[RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+ by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+ with ESMTP id dV48znJzOCV6 for <notmuch@notmuchmail.org>;\r
+ Fri, 27 Jun 2014 12:37:06 -0700 (PDT)\r
+Received: from dmz-mailsec-scanner-7.mit.edu (dmz-mailsec-scanner-7.mit.edu\r
+ [18.7.68.36])\r
+ (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))\r
+ (No client certificate requested)\r
+ by olra.theworths.org (Postfix) with ESMTPS id 4D821431FAE\r
+ for <notmuch@notmuchmail.org>; Fri, 27 Jun 2014 12:37:06 -0700 (PDT)\r
+X-AuditID: 12074424-f79146d00000067c-c9-53adc7e18bdc\r
+Received: from mailhub-auth-1.mit.edu ( [18.9.21.35])\r
+ (using TLS with cipher AES256-SHA (256/256 bits))\r
+ (Client did not present a certificate)\r
+ by dmz-mailsec-scanner-7.mit.edu (Symantec Messaging Gateway) with SMTP\r
+ id 02.9A.01660.1E7CDA35; Fri, 27 Jun 2014 15:37:05 -0400 (EDT)\r
+Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11])\r
+ by mailhub-auth-1.mit.edu (8.13.8/8.9.2) with ESMTP id s5RJb2RE011699; \r
+ Fri, 27 Jun 2014 15:37:03 -0400\r
+Received: from awakening.csail.mit.edu (awakening.csail.mit.edu [18.26.4.91])\r
+ (authenticated bits=0)\r
+ (User authenticated as amdragon@ATHENA.MIT.EDU)\r
+ by outgoing.mit.edu (8.13.8/8.12.4) with ESMTP id s5RJb0R1013147\r
+ (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT);\r
+ Fri, 27 Jun 2014 15:37:01 -0400\r
+Received: from amthrax by awakening.csail.mit.edu with local (Exim 4.80)\r
+ (envelope-from <amdragon@mit.edu>)\r
+ id 1X0bxD-0002cV-Qt; Fri, 27 Jun 2014 15:36:59 -0400\r
+Date: Fri, 27 Jun 2014 15:36:59 -0400\r
+From: Austin Clements <amdragon@MIT.EDU>\r
+To: Jakub Wilk <jwilk@debian.org>\r
+Subject: Re: Bug#749890: python3-notmuch: missing header in mbox message ->\r
+ NullPointerError\r
+Message-ID: <20140627193659.GH4660@mit.edu>\r
+References: <8738ewudra.fsf@zancas.localnet> <20140623201918.GA7346@jwilk.net>\r
+ <87ha37fjm3.fsf@zancas.localnet> <20140626213100.GA8930@jwilk.net>\r
+ <878uoifj9n.fsf@zancas.localnet>\r
+MIME-Version: 1.0\r
+Content-Type: text/plain; charset=us-ascii\r
+Content-Disposition: inline\r
+In-Reply-To: <878uoifj9n.fsf@zancas.localnet>\r
+User-Agent: Mutt/1.5.21 (2010-09-15)\r
+X-Brightmail-Tracker:\r
+ H4sIAAAAAAAAA+NgFlrJKsWRmVeSWpSXmKPExsUixCmqrPvw+Npgg6ZXKhZbXhVa3GjtZrT4\r
+ NP8Qu8X1mzOZHVg8LmydxOLxq20us8ezVbeYPbYces8cwBLFZZOSmpNZllqkb5fAlXH5Wztj\r
+ wRuxir0Tm5gaGJcJdTFyckgImEhM3DqJCcIWk7hwbz1bFyMXh5DAbCaJZTuvsEA4Gxkllu89\r
+ zgThnGaSuHn0HTuEs4RR4vrKHhaQfhYBVYnFm1eygdhsAhoS2/YvZwSxRQQUJY4e7GMGsZkF\r
+ 4iVWznkDZgsLxEnsX9fH2sXIwcEroC3xZastxMxdjBJTH6xnB6nhFRCUODnzCQtEr5bEjX8v\r
+ mUDqmQWkJZb/4wAJcwroSrx78h5spKiAisSUk9vYJjAKzULSPQtJ9yyE7gWMzKsYZVNyq3Rz\r
+ EzNzilOTdYuTE/PyUot0zfVyM0v0UlNKNzGCwp/dRWUHY/MhpUOMAhyMSjy8np1rg4VYE8uK\r
+ K3MPMUpyMCmJ8i49AhTiS8pPqcxILM6ILyrNSS0+xCjBwawkwiu9AijHm5JYWZValA+TkuZg\r
+ URLnfWttFSwkkJ5YkpqdmlqQWgSTleHgUJLgFQLGuZBgUWp6akVaZk4JQpqJgxNkOA/QcEWQ\r
+ Gt7igsTc4sx0iPwpRmOOT9eOtTFxPJp0qo1JiCUvPy9VSpxX9BhQqQBIaUZpHtw0WAp7xSgO\r
+ 9Jww7z+QKh5g+oOb9wpoFRPQKvOCVSCrShIRUlINjJ3KPw8dCpL8+/1u2nXfGfs3BRhOKuG9\r
+ KuQvX3c4QWiVzVn5C14ZrL25dx9dif6tyr6zX6zt/8Uww73cHOtOTWjr3eB02oR33Z+DXnpn\r
+ 14r/+Lt8qd98/icCjXobmCQVrFXFAuxY7tsnPeBkW8l259oUxbkpJx8atuevOvhYom7TpbK0\r
+ m70OJ5RYijMSDbWYi4oTAZLnIAs8AwAA\r
+Cc: notmuch@notmuchmail.org, 749890@bugs.debian.org\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+ <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Fri, 27 Jun 2014 19:37:12 -0000\r
+\r
+Quoth David Bremner on Jun 27 at 12:45 pm:\r
+> Jakub Wilk <jwilk@debian.org> writes:\r
+> \r
+> > * David Bremner <david@tethera.net>, 2014-06-26, 18:26:\r
+> >>>0.18.1~rc0-1 is much better, thanks!\r
+> >>>\r
+> >>>I still get NullPointerError for one of my messages, though. :-( The \r
+> >>>message is in the MBOXCL format (where message body size is indicated \r
+> >>>by the Content-Length field), and has lines starting with "From " in \r
+> >>>the message body. I've attached a new test case.\r
+> >>\r
+> >>That message (and at a guess other MBOXCL files) is ignored as a \r
+> >>non-mail file by 0.18.1 "notmuch new".\r
+> >\r
+> > Indeed.\r
+> >\r
+> >>Is this another case of files which where indexed with an older version \r
+> >>of notmuch causing problems with a newer version?\r
+> >\r
+> > Yes, that's why I meant. Sorry for not being clear.\r
+> \r
+> As a point of information, I bisected with the following test script:\r
+> \r
+> #!/usr/bin/env bash\r
+> test_description='"notmuch new" in several variations'\r
+> . ./test-lib.sh\r
+> \r
+> test_begin_subtest "Support single-message mbox with content length (deprecated)"\r
+> cat > "${MAIL_DIR}"/mbox_file2 <<EOF\r
+> From jwilk Fri May 30 14:09:05 2014\r
+> Subject: Hello world!\r
+> Content-Length: 12\r
+> Lines: 1\r
+> \r
+> From world!\r
+> \r
+> EOF\r
+> output=$(NOTMUCH_NEW 2>&1)\r
+> test_expect_equal "$output" \\r
+> "Added 1 new message to the database."\r
+> \r
+> \r
+> test_done\r
+> \r
+> The commit where the behaviour changed to reject MBOXCL files with \r
+> 'From ' in the body was 610f0e09929. This was between 0.14 and 0.15.\r
+> I'd say this was unintentional, although it isn't clear to me yet how\r
+> easy it is fix.\r
+\r
+Thanks for bisecting this, David.\r
+\r
+Unfortunately, when it comes to mbox, the only winning move is not to\r
+play.\r
+\r
+The reason 610f0e09929 matters here is because it *added* support for\r
+mbox (or, rather, this weird but surprisingly common chimera of\r
+mbox-formatted message files with maildir-formatted file names).\r
+Previously, notmuch assumed *everything* was a maildir-formatted\r
+message file; that is, one message per file. It "worked" for mboxcl\r
+because it had no idea what either mbox or mboxcl was. But it would\r
+choke hard when it encountered a large, multi-message mbox archive\r
+because it would try to index the whole thing as one giant email. In\r
+an effort to avoid this, I added explicit support for single-message\r
+mbox files (to keep the chimerians happy). But at that point we lost:\r
+there simply is no way to reliably and programmatically distinguish\r
+the many variants of mbox (see\r
+http://www.jwz.org/doc/content-length.html for a good discussion of\r
+this).\r
+\r
+So, I'm afraid my best advice is to convert your mboxcl files to\r
+something else. Probably maildir, both because you're storing them in\r
+a maildir (I assume?) and because it's easy: just strip off the first\r
+line. I don't think there's anything notmuch can do to fix this\r
+without breaking something else.\r
projects / notmuch-archives.git / commitdiff