notmuch new: emails with BOM are ignored

diff --git a/3d/f9d47984540d3a23d02b74ec90f636e2f6cbc6 b/3d/f9d47984540d3a23d02b74ec90f636e2f6cbc6
new file mode 100644 (file)
index 0000000..dab7aec
--- /dev/null
+++ b/3d/f9d47984540d3a23d02b74ec90f636e2f6cbc6
@@ -0,0 +1,104 @@
+Return-Path: <a.amann@ucc.ie>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+       by olra.theworths.org (Postfix) with ESMTP id D57A6431FBD\r
+       for <notmuch@notmuchmail.org>; Tue, 27 May 2014 01:50:41 -0700 (PDT)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -0.7\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-0.7 tagged_above=-999 required=5\r
+       tests=[RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+       by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+       with ESMTP id xRJiPUwK6stO for <notmuch@notmuchmail.org>;\r
+       Tue, 27 May 2014 01:50:34 -0700 (PDT)\r
+X-Greylist: delayed 892 seconds by postgrey-1.32 at olra;\r
+       Tue, 27 May 2014 01:50:34 PDT\r
+Received: from emea01-am1-obe.outbound.protection.outlook.com\r
+       (mail-am1lp0017.outbound.protection.outlook.com [213.199.154.17])\r
+       (using TLSv1 with cipher AES128-SHA (128/128 bits))\r
+       (No client certificate requested)\r
+       by olra.theworths.org (Postfix) with ESMTPS id 45BC5431FAF\r
+       for <notmuch@notmuchmail.org>; Tue, 27 May 2014 01:50:34 -0700 (PDT)\r
+Received: from AM1FFO11FD011.protection.gbl (10.174.64.31) by\r
+       AM1FFO11HUB010.protection.gbl (10.174.64.138) with Microsoft SMTP\r
+       Server (TLS) id 15.0.949.9; Tue, 27 May 2014 08:35:37 +0000\r
+Received: from mail4.ucc.ie (143.239.1.24) by\r
+       AM1FFO11FD011.mail.protection.outlook.com (10.174.65.100) with\r
+       Microsoft SMTP Server (TLS) id 15.0.949.9 via Frontend Transport;\r
+       Tue, 27 May 2014 08:35:37 +0000\r
+Received: from msstf091.ucc.ie (msstf091.ucc.ie [143.239.76.91])\r
+       by mail4.ucc.ie (8.14.4/8.14.4) with ESMTP id s4R8ZYon020133\r
+       for <notmuch@notmuchmail.org>; Tue, 27 May 2014 09:35:36 +0100\r
+Received: by msstf091.ucc.ie (Postfix, from userid 1000)\r
+       id CBCCEA2E13; Tue, 27 May 2014 09:35:33 +0100 (IST)\r
+From: Andreas Amann <a.amann@ucc.ie>\r
+To: <notmuch@notmuchmail.org>\r
+Subject: notmuch new: emails with BOM are ignored\r
+User-Agent: Notmuch/0.18 (http://notmuchmail.org) Emacs/24.3.1\r
+       (x86_64-unknown-linux-gnu)\r
+Date: Tue, 27 May 2014 09:35:33 +0100\r
+Message-ID: <874n0b4o16.fsf@msstf091.ucc.ie>\r
+MIME-Version: 1.0\r
+Content-Type: text/plain\r
+X-EOPAttributedMessage: 0\r
+X-Forefront-Antispam-Report: CIP:143.239.1.24; CTRY:IE; IPV:NLI; IPV:NLI;\r
+       EFV:NLI; SFV:NSPM;\r
+       SFS:(6009001)(438001)(189002)(199002)(80022001)(20776003)(6806004)(76482001)(81542001)(48376002)(83322001)(81342001)(19580395003)(45336002)(64706001)(79102001)(40036003)(50466002)(44976005)(77982001)(92726001)(54356999)(86362001)(87936001)(102836001)(50986999)(74502001)(92566001)(4396001)(74662001)(74482001)(31966008)(21056001)(85852003)(83506001)(83072002)(46102001)(42186004)(46386002)(99396002)(36756003)(33646001)(52956003)(90966001);        DIR:OUT; SFP:; SCL:1; SRVR:AM1FFO11HUB010; H:mail4.ucc.ie; FPR:;        MLV:sfv;\r
+ PTR:mail4.ucc.ie; MX:1; A:0; LANG:en; \r
+X-OriginatorOrg: ucc.ie\r
+X-Forefront-PRVS: 02243C58C6\r
+Received-SPF: Pass (: domain of ucc.ie designates 143.239.1.24 as permitted\r
+       sender) receiver=; client-ip=143.239.1.24; helo=mail4.ucc.ie;\r
+Authentication-Results: spf=pass (sender IP is 143.239.1.24)\r
+       smtp.mailfrom=a.amann@ucc.ie; \r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+       <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Tue, 27 May 2014 08:50:42 -0000\r
+\r
+Hi,\r
+\r
+I recently received some spam mails, which have a utf-16 byte order mark\r
+(BOM) U+FEFF as the first character in one of their "Received:"\r
+lines. When I run "notmuch new" I get the following:\r
+\r
+Note: Ignoring non-mail file: /home/user/Mail/new/path_to_email_with_BOM\r
+\r
+Could this be a bug in notmuch? Possible emails with BOM are not "legal"\r
+anyhow, but in my opinion it would be better to simply ignore the BOM.\r
+\r
+I am using notmuch 0.18.\r
+\r
+Andreas\r
+\r
+\r
+\r
+P.S.: \r
+\r
+looking quickly at the code as a non-expert, it seems that the error\r
+message comes from function "add_file" in "notmuch-new.c", because it\r
+gets a status of "NOTMUCH_STATUS_FILE_NOT_EMAIL" from\r
+"notmuch_database_add_message". This latter function is defined in\r
+"lib/database.cc". It returns that status in the case, when\r
+"notmuch_message_file_get_header" does not work (i.e. return NULL or "")\r
+for the headers "from", "subject" or "to".  In the message at hand all\r
+of these headers only come after the BOM, so my guess is that\r
+"notmuch_message_file_get_header" somehow chokes when it hits on the\r
+BOM?\r
+\r
+\r
+\r
+\r
+\r