[PATCH 2/2] Make parsing of References and In-Reply-To header less error prone

diff --git a/16/36ef3e028388a499907e13eae0fa853dae0aa0 b/16/36ef3e028388a499907e13eae0fa853dae0aa0
new file mode 100644 (file)
index 0000000..95fc6c2
--- /dev/null
+++ b/16/36ef3e028388a499907e13eae0fa853dae0aa0
@@ -0,0 +1,128 @@
+Return-Path: <sojkam1@fel.cvut.cz>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+       by olra.theworths.org (Postfix) with ESMTP id EB92E431FBD\r
+       for <notmuch@notmuchmail.org>; Mon, 26 May 2014 07:06:18 -0700 (PDT)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -2.3\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-2.3 tagged_above=-999 required=5\r
+       tests=[RCVD_IN_DNSWL_MED=-2.3] autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+       by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+       with ESMTP id qn2HFwlTKUBV for <notmuch@notmuchmail.org>;\r
+       Mon, 26 May 2014 07:06:10 -0700 (PDT)\r
+Received: from max.feld.cvut.cz (max.feld.cvut.cz [147.32.192.36])\r
+       by olra.theworths.org (Postfix) with ESMTP id 8B123431FAF\r
+       for <notmuch@notmuchmail.org>; Mon, 26 May 2014 07:06:10 -0700 (PDT)\r
+Received: from localhost (unknown [192.168.200.7])\r
+       by max.feld.cvut.cz (Postfix) with ESMTP id 191E319F3634\r
+       for <notmuch@notmuchmail.org>; Mon, 26 May 2014 16:06:06 +0200 (CEST)\r
+X-Virus-Scanned: IMAP STYX AMAVIS\r
+Received: from max.feld.cvut.cz ([192.168.200.1])\r
+       by localhost (styx.feld.cvut.cz [192.168.200.7]) (amavisd-new,\r
+       port 10044) with ESMTP id MGCC8KNkeYLG for <notmuch@notmuchmail.org>;\r
+       Mon, 26 May 2014 16:06:02 +0200 (CEST)\r
+Received: from imap.feld.cvut.cz (imap.feld.cvut.cz [147.32.192.34])\r
+       by max.feld.cvut.cz (Postfix) with ESMTP id F07CE19F362F\r
+       for <notmuch@notmuchmail.org>; Mon, 26 May 2014 16:06:01 +0200 (CEST)\r
+Received: from wsh by steelpick.2x.cz with local (Exim 4.82)\r
+       (envelope-from <sojkam1@fel.cvut.cz>)\r
+       id 1WovXM-0008SE-1z; Mon, 26 May 2014 16:06:00 +0200\r
+From: Michal Sojka <sojkam1@fel.cvut.cz>\r
+To: notmuch@notmuchmail.org\r
+Subject: [PATCH 2/2] Make parsing of References and In-Reply-To header less\r
+       error prone\r
+Date: Mon, 26 May 2014 16:05:57 +0200\r
+Message-Id: <1401113157-32454-3-git-send-email-sojkam1@fel.cvut.cz>\r
+X-Mailer: git-send-email 2.0.0.rc2\r
+In-Reply-To: <1401113157-32454-1-git-send-email-sojkam1@fel.cvut.cz>\r
+References: <1401113157-32454-1-git-send-email-sojkam1@fel.cvut.cz>\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+       <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Mon, 26 May 2014 14:06:19 -0000\r
+\r
+According to RFC2822 References and In-Reply-To headers are supposed\r
+to contain one or more Message-IDs, however older RFC822 allowed\r
+almost any content. When both References and In-Reply-To headers ends\r
+with something else that a Message-ID (see e.g. [1]), the thread\r
+structure presented by notmuch is incorrect. The reason is that\r
+notmuch treats this case as if the email contained no "replyto"\r
+information (see _notmuch_database_link_message_to_parents).\r
+\r
+This patch changes the parse_references() function to return the last\r
+valid Message-ID encountered rather than NULL resulting from the last\r
+hunk of text not being the Message-ID.\r
+\r
+[1] https://lkml.org/lkml/headers/2014/5/19/864\r
+---\r
+ lib/database.cc             | 15 ++++++---------\r
+ test/T510-thread-replies.sh |  1 -\r
+ 2 files changed, 6 insertions(+), 10 deletions(-)\r
+\r
+diff --git a/lib/database.cc b/lib/database.cc\r
+index 1efb14d..26f15fd 100644\r
+--- a/lib/database.cc\r
++++ b/lib/database.cc\r
+@@ -521,7 +521,7 @@ parse_references (void *ctx,\r
+                 GHashTable *hash,\r
+                 const char *refs)\r
+ {\r
+-    char *ref;\r
++    char *ref, *last_ref = NULL;\r
+ \r
+     if (refs == NULL || *refs == '\0')\r
+       return NULL;\r
+@@ -529,20 +529,17 @@ parse_references (void *ctx,\r
+     while (*refs) {\r
+       ref = _parse_message_id (ctx, refs, &refs);\r
+ \r
+-      if (ref && strcmp (ref, message_id))\r
++      if (ref && strcmp (ref, message_id)) {\r
+           g_hash_table_insert (hash, ref, NULL);\r
++          last_ref = ref;\r
++      }\r
+     }\r
+ \r
+     /* The return value of this function is used to add a parent\r
+      * reference to the database.  We should avoid making a message\r
+-     * its own parent, thus the following check.\r
++     * its own parent, thus the above check.\r
+      */\r
+-\r
+-    if (ref && strcmp(ref, message_id)) {\r
+-      return ref;\r
+-    } else {\r
+-      return NULL;\r
+-    }\r
++    return last_ref;\r
+ }\r
+ \r
+ notmuch_status_t\r
+diff --git a/test/T510-thread-replies.sh b/test/T510-thread-replies.sh\r
+index d818b89..1392fbe 100755\r
+--- a/test/T510-thread-replies.sh\r
++++ b/test/T510-thread-replies.sh\r
+@@ -138,7 +138,6 @@ expected=`echo "$expected" | notmuch_json_show_sanitize`\r
+ test_expect_equal_json "$output" "$expected"\r
+ \r
+ test_begin_subtest "Ignore garbage at the end of References"\r
+-test_subtest_known_broken\r
+ add_message '[id]="foo@five.com"' \\r
+     '[subject]="five"'\r
+ add_message '[id]="bar@five.com"' \\r
+-- \r
+2.0.0.rc2\r
+\r