[PATCH 00/11] Add ghost messages and fix thread linking

diff --git a/54/80b2f6a5163db6b92370d1a81a582cffcd3d1d b/54/80b2f6a5163db6b92370d1a81a582cffcd3d1d
new file mode 100644 (file)
index 0000000..b97a445
--- /dev/null
+++ b/54/80b2f6a5163db6b92370d1a81a582cffcd3d1d
@@ -0,0 +1,91 @@
+Return-Path: <aclements@csail.mit.edu>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+       by olra.theworths.org (Postfix) with ESMTP id 01512431FDA\r
+       for <notmuch@notmuchmail.org>; Fri,  3 Oct 2014 07:19:32 -0700 (PDT)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -2.3\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-2.3 tagged_above=-999 required=5\r
+       tests=[RCVD_IN_DNSWL_MED=-2.3] autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+       by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+       with ESMTP id ogxsEZTJupP7 for <notmuch@notmuchmail.org>;\r
+       Fri,  3 Oct 2014 07:19:25 -0700 (PDT)\r
+Received: from dmz-mailsec-scanner-4.mit.edu (dmz-mailsec-scanner-4.mit.edu\r
+       [18.9.25.15])\r
+       (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))\r
+       (No client certificate requested)\r
+       by olra.theworths.org (Postfix) with ESMTPS id 8D5C8431FC0\r
+       for <notmuch@notmuchmail.org>; Fri,  3 Oct 2014 07:19:25 -0700 (PDT)\r
+X-AuditID: 1209190f-f79aa6d000005b45-a6-542eb06b895a\r
+Received: from mailhub-auth-3.mit.edu ( [18.9.21.43])\r
+       (using TLS with cipher AES256-SHA (256/256 bits))\r
+       (Client did not present a certificate)\r
+       by dmz-mailsec-scanner-4.mit.edu (Symantec Messaging Gateway) with SMTP\r
+       id 3B.24.23365.B60BE245; Fri,  3 Oct 2014 10:19:23 -0400 (EDT)\r
+Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11])\r
+       by mailhub-auth-3.mit.edu (8.13.8/8.9.2) with ESMTP id s93EJNx7018546\r
+       for <notmuch@notmuchmail.org>; Fri, 3 Oct 2014 10:19:23 -0400\r
+Received: from drake.dyndns.org\r
+       (216-15-114-40.c3-0.arl-ubr1.sbo-arl.ma.cable.rcn.com\r
+       [216.15.114.40]) (authenticated bits=0)\r
+       (User authenticated as amdragon@ATHENA.MIT.EDU)\r
+       by outgoing.mit.edu (8.13.8/8.12.4) with ESMTP id s93EJLC0023304\r
+       (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT)\r
+       for <notmuch@notmuchmail.org>; Fri, 3 Oct 2014 10:19:23 -0400\r
+Received: from amthrax by drake.dyndns.org with local (Exim 4.84)\r
+       (envelope-from <aclements@csail.mit.edu>) id 1Xa3hY-0002JV-Tt\r
+       for notmuch@notmuchmail.org; Fri, 03 Oct 2014 10:19:20 -0400\r
+From: Austin Clements <aclements@csail.mit.edu>\r
+To: notmuch@notmuchmail.org\r
+Subject: [PATCH 00/11] Add ghost messages and fix thread linking\r
+Date: Fri,  3 Oct 2014 10:19:07 -0400\r
+Message-Id: <1412345958-8278-1-git-send-email-aclements@csail.mit.edu>\r
+X-Mailer: git-send-email 2.1.0\r
+X-Brightmail-Tracker:\r
+ H4sIAAAAAAAAA+NgFprNIsWRmVeSWpSXmKPExsUixCmqrZu9QS/E4GKbjcX1mzOZHRg9nq26\r
+       xRzAGMVlk5Kak1mWWqRvl8CV0X3yMGvBGvaKB4t3sTQwdrJ1MXJySAiYSNx9+Z8VwhaTuHBv\r
+       PVCci0NIYDaTxN2Od8wgCSGBc4wSF5+FQiR+Mkl0nfvDAuEcYpT4//IgWBWbgL7EirWTwEaJ\r
+       CEhL7Lw7G8wWFnCQ2HD7MQuIzSKgKrF91VGwel4BN4l7f++zQ6yWk9iw+z/jBEaeBYwMqxhl\r
+       U3KrdHMTM3OKU5N1i5MT8/JSi3RN9HIzS/RSU0o3MYL9Lcm/g/HbQaVDjAIcjEo8vB9u6IYI\r
+       sSaWFVfmHmKU5GBSEuWtXKwXIsSXlJ9SmZFYnBFfVJqTWgx0MQezkgjvgpVAOd6UxMqq1KJ8\r
+       mJQ0B4uSOO+mH3whQgLpiSWp2ampBalFMFkZDg4lCd729UCNgkWp6akVaZk5JQhpJg5OkOE8\r
+       QMPBaniLCxJzizPTIfKnGHU5Wpre9jIJseTl56VKifM+XAdUJABSlFGaBzcHFqevGMWB3hLm\r
+       ZQJGrRAPMMbhJr0CWsIEtOSdvS7IkpJEhJRUA+OECMHH+0VLw0WdlPQdOrh/acbES515n8/P\r
+       /P7xepnpnRVvHNolZm9xFu/Ta9CwWK1x9ebmRKf/2rtcNmw5WPj1YsSxvrVOfV2bZkYccXec\r
+       LH6aZ1beWqew782Gpa5TWOY9z2NjqTT0Z5lzUuuQgs0nwYlKeT8/ePrcaFNsfJDY6bhiztob\r
+       /5RYijMSDbWYi4oTAeQvl6SuAgAA\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+       <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Fri, 03 Oct 2014 14:19:32 -0000\r
+\r
+This series modifies our database representation of messages that have\r
+been referenced by other messages, but for which we don't have the\r
+message itself.  Currently, we store this information as Xapian\r
+metadata, but this has several downsides for performance and\r
+complexity and results in hard-to-fix thread linking bugs.  This patch\r
+series implements "ghost messages", which replace this Xapian metadata\r
+with Xapian documents that look and act very much like regular message\r
+documents, but simply have no content.  This simplifies and speeds up\r
+our thread linking algorithm and fixes the currently broken thread\r
+linking test.\r
+\r
+Ghost messages also open up interesting future possibilities, such as\r
+"pre-seeding" tags for messages that are not yet indexed.  This could\r
+be used to make notmuch insert simpler and more robust, as part of tag\r
+synchronization, and to improve nmbug's behavior when tags arrive\r
+before messages.\r
+\r