[notmuch] Missing messages breaking threads

diff --git a/a4/8c0e320c1a365dbaf3f3955d12d044b5524ab9 b/a4/8c0e320c1a365dbaf3f3955d12d044b5524ab9
new file mode 100644 (file)
index 0000000..3ba4971
--- /dev/null
+++ b/a4/8c0e320c1a365dbaf3f3955d12d044b5524ab9
@@ -0,0 +1,107 @@
+Return-Path: <james@jameswestby.net>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+       by olra.theworths.org (Postfix) with ESMTP id 2FFC2431FAE\r
+       for <notmuch@notmuchmail.org>; Fri, 18 Dec 2009 11:02:31 -0800 (PST)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+       by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+       with ESMTP id EPAqHHbqLDbF for <notmuch@notmuchmail.org>;\r
+       Fri, 18 Dec 2009 11:02:30 -0800 (PST)\r
+Received: from mout.perfora.net (mout.perfora.net [74.208.4.195])\r
+       by olra.theworths.org (Postfix) with ESMTP id 6C5F8431FC3\r
+       for <notmuch@notmuchmail.org>; Fri, 18 Dec 2009 11:02:30 -0800 (PST)\r
+Received: from jameswestby.net (jameswestby.net [89.145.97.141])\r
+       by mx.perfora.net (node=mxus2) with ESMTP (Nemesis)\r
+       id 0MA8Qr-1NAThE1ykT-00BOWU for notmuch@notmuchmail.org;\r
+       Fri, 18 Dec 2009 14:02:29 -0500\r
+Received: from cpc4-aztw22-2-0-cust59.aztw.cable.virginmedia.com\r
+       ([94.169.116.60] helo=flash)\r
+       by jameswestby.net with esmtpa (Exim 4.69)\r
+       (envelope-from <james@jameswestby.net>) id 1NLi5r-0005oR-5z\r
+       for notmuch@notmuchmail.org; Fri, 18 Dec 2009 19:02:27 +0000\r
+Received: by flash (Postfix, from userid 1000)\r
+       id 90CFA6E546A; Fri, 18 Dec 2009 19:02:21 +0000 (GMT)\r
+From: James Westby <jw+debian@jameswestby.net>\r
+To: notmuch@notmuchmail.org\r
+Date: Fri, 18 Dec 2009 19:02:21 +0000\r
+Message-ID: <87oclwrtqa.fsf@jameswestby.net>\r
+MIME-Version: 1.0\r
+Content-Type: text/plain; charset=us-ascii\r
+Subject: [notmuch] Missing messages breaking threads\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.12\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+       <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Fri, 18 Dec 2009 19:02:31 -0000\r
+\r
+Hi,\r
+\r
+I like the architecture of notmuch, and have just switched\r
+to using it as my primary client, so thanks.\r
+\r
+I however have discovered one issue that is a pain.\r
+\r
+I use a bugtracker a lot which has workflows that mean that\r
+you don't always get teh initial messages from a bug.\r
+\r
+To put it in more common terms, imagine this:\r
+\r
+  * Alice sends a mail to a large group of your friends, but\r
+    not you.\r
+  * Each of these friends replies, and puts you in Cc for the\r
+    reply.\r
+\r
+This will mean that you get several messages that all have\r
+References and In-Reply-To set to ids that aren't known to\r
+notmuch.\r
+\r
+This means that it doesn't thread them, and so they aren't\r
+grouped in the UI. This becomes painful when you are dealing\r
+with several bugs like this. It's almost like being back in\r
+the days of a message oriented client, and we know that's\r
+not the way to do it.\r
+\r
+Therefore I'd like to fix this. The obvious way is to\r
+introduce documents in to the db for each id we see, and\r
+threading should then naturally work better.\r
+\r
+The only issue I see with doing this is with mail delays.\r
+Once we do this we will sometimes receive a message that\r
+already has a dummy document. What happens currently with\r
+message-id collisions?\r
+\r
+Therefore I would propose this:\r
+\r
+  * When doing a thread resolution and we have ids that\r
+    we don't know, add a document to the db that is\r
+    something like\r
+\r
+    {id: <message-id>\r
+     thread: <synthesised thread-id>\r
+     dummy: True\r
+     content: "Messages missing"\r
+    }\r
+\r
+  * When we get a message-id conflict check for dummy:True\r
+    and replace the document if it is there.\r
+\r
+How does this sound?\r
+\r
+There could be an issue with synthesising too many threads\r
+and then ending up having to try and put a message in two\r
+threads? I see there is code for merging threads, would that\r
+handle this?\r
+\r
+Thanks,\r
+\r
+James\r