How to debug 'ignoring non-mail file' issues

diff --git a/2e/d8e60e1bdae9addaa32af9faaa2e36d48a940a b/2e/d8e60e1bdae9addaa32af9faaa2e36d48a940a
new file mode 100644 (file)
index 0000000..c86c90c
--- /dev/null
+++ b/2e/d8e60e1bdae9addaa32af9faaa2e36d48a940a
@@ -0,0 +1,90 @@
+Return-Path: <perttu.luukko@iki.fi>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+       by olra.theworths.org (Postfix) with ESMTP id 2E131431FBD\r
+       for <notmuch@notmuchmail.org>; Sun, 31 Aug 2014 00:41:20 -0700 (PDT)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: 0\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=0 tagged_above=-999 required=5 tests=[none]\r
+       autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+       by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+       with ESMTP id SgODyZheZYV4 for <notmuch@notmuchmail.org>;\r
+       Sun, 31 Aug 2014 00:41:15 -0700 (PDT)\r
+Received: from mail.kapsi.fi (mail.kapsi.fi [217.30.184.167])\r
+       (using TLSv1 with cipher AES256-SHA (256/256 bits))\r
+       (No client certificate requested)\r
+       by olra.theworths.org (Postfix) with ESMTPS id 6D074431FB6\r
+       for <notmuch@notmuchmail.org>; Sun, 31 Aug 2014 00:41:15 -0700 (PDT)\r
+Received: from dsl-jklbrasgw1-54fb12-136.dhcp.inet.fi ([84.251.18.136]\r
+       helo=localhost)\r
+       by mail.kapsi.fi with esmtpsa (TLS1.0:RSA_AES_128_CBC_SHA1:16)\r
+       (Exim 4.72) (envelope-from <perttu.luukko@iki.fi>)\r
+       id 1XNzl1-0001m9-4R\r
+       for notmuch@notmuchmail.org; Sun, 31 Aug 2014 10:41:03 +0300\r
+Date: Sun, 31 Aug 2014 10:40:59 +0300\r
+From: Perttu Luukko <perttu.luukko@iki.fi>\r
+To: notmuch@notmuchmail.org\r
+Subject: How to debug 'ignoring non-mail file' issues\r
+Message-ID: <20140831074059.GA13399@rakuuna.dhcp.inet.fi>\r
+MIME-Version: 1.0\r
+Content-Type: text/plain; charset=us-ascii\r
+Content-Disposition: inline\r
+User-Agent: Mutt/1.5.22.1 (2013-10-16)\r
+X-SA-Exim-Connect-IP: 84.251.18.136\r
+X-SA-Exim-Mail-From: perttu.luukko@iki.fi\r
+X-SA-Exim-Scanned: No (on mail.kapsi.fi); SAEximRunCond expanded to false\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+       <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Sun, 31 Aug 2014 07:41:20 -0000\r
+\r
+Hi,\r
+\r
+I indexed my archive of emails from recent years with notmuch (about 10k\r
+messages so not much). I have quite a lot of messages 'notmuch new'\r
+ignores as non-mail files, about 1000 of them. They are not obviously\r
+malformed, meaning that the files certainly look like emails when opened\r
+in a text editor. I'd like to find out why these files are ignored, and\r
+if there is something I can do to fix them. Of course I'd like to have a\r
+complete database of my old emails, with nothing falling through the\r
+cracks like this.\r
+\r
+The vast majority of these ignored mails are not ignored after I\r
+transfer them with offlineimap to another computer. I can non-ignore\r
+these files probably by copying the renamed file back to the mail\r
+server, so this is fixable. Offlineimap shouldn't mess with the file's\r
+contents, so is there something that can cause notmuch to ignore a file\r
+based on its name?\r
+\r
+Looking at the rest of the ignored messages most of them seem to have\r
+very large attachments, but there are possibly others. There is only\r
+maybe 20 of these kinds of emails so I can try to fix them manually.\r
+Still, it would help if I knew what exactly caused notmuch to ignore the\r
+file. I understand most of the message parsing is done with gmime. Does\r
+gmime give any diagnostics on parse errors that could be used to give a\r
+reason for thinking a file is not mail?\r
+\r
+I understand that the list of non-mail files is stored in the notmuch\r
+database and the files are completely ignored from there on. This\r
+actually makes it harder to debug these kind of issues since the list of\r
+ignored mails is only visible on the first invocation of 'notmuch new',\r
+unless the files are moved around. Is there some way to extract the list\r
+of ignored files from the database for inspection? Maybe 'notmuch new'\r
+could have some kind of --unignore-non-mail switch that would reconsider\r
+previously ignored files.\r
+\r
+-- \r
+Perttu Luukko\r