Bug report: undeletable traces of ghosts in the notmuch database

diff --git a/6a/2171a3f7a10e36140ec23d51fee60e8ddf8454 b/6a/2171a3f7a10e36140ec23d51fee60e8ddf8454
new file mode 100644 (file)
index 0000000..bea520f
--- /dev/null
+++ b/6a/2171a3f7a10e36140ec23d51fee60e8ddf8454
@@ -0,0 +1,85 @@
+Return-Path: <dkg@fifthhorseman.net>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+ by arlo.cworth.org (Postfix) with ESMTP id 6C4716DE00DD\r
+ for <notmuch@notmuchmail.org>; Thu, 14 Jan 2016 13:21:14 -0800 (PST)\r
+X-Virus-Scanned: Debian amavisd-new at cworth.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -0.026\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-0.026 tagged_above=-999 required=5\r
+ tests=[AWL=-0.026] autolearn=disabled\r
+Received: from arlo.cworth.org ([127.0.0.1])\r
+ by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024)\r
+ with ESMTP id 5EKYVS4Bg6Rb for <notmuch@notmuchmail.org>;\r
+ Thu, 14 Jan 2016 13:21:12 -0800 (PST)\r
+Received: from che.mayfirst.org (che.mayfirst.org [209.234.253.108])\r
+ by arlo.cworth.org (Postfix) with ESMTP id 9FC546DE1607\r
+ for <notmuch@notmuchmail.org>; Thu, 14 Jan 2016 13:21:12 -0800 (PST)\r
+Received: from fifthhorseman.net (unknown [38.109.115.130])\r
+ by che.mayfirst.org (Postfix) with ESMTPSA id 66047F984\r
+ for <notmuch@notmuchmail.org>; Thu, 14 Jan 2016 16:21:00 -0500 (EST)\r
+Received: by fifthhorseman.net (Postfix, from userid 1000)\r
+ id CD5F620000; Thu, 14 Jan 2016 13:20:59 -0800 (PST)\r
+From: Daniel Kahn Gillmor <dkg@fifthhorseman.net>\r
+To: notmuch@notmuchmail.org\r
+Subject: Bug report: undeletable traces of ghosts in the notmuch database\r
+User-Agent: Notmuch/0.21+66~g8c19a9a (http://notmuchmail.org) Emacs/24.5.1\r
+ (x86_64-pc-linux-gnu)\r
+Date: Thu, 14 Jan 2016 16:20:59 -0500\r
+Message-ID: <87twmf7t8k.fsf@alice.fifthhorseman.net>\r
+MIME-Version: 1.0\r
+Content-Type: text/plain\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.20\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+ <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <https://notmuchmail.org/mailman/options/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch/>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <https://notmuchmail.org/mailman/listinfo/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Thu, 14 Jan 2016 21:21:14 -0000\r
+\r
+If i add a message to my notmuch database that includes References: or\r
+In-Reply-To: of a message-id that i don't have, notmuch creates a\r
+"ghost" entry in the database.\r
+\r
+If i remove the message from my mail archive, and then re-run "notmuch\r
+new", i'd expect all traces of that message to be gone completely.\r
+\r
+However, the ghost message-IDs appear to remain in the database.  I've\r
+tested this by:\r
+\r
+ * create a demo notmuch installation\r
+\r
+ * use delve -a to see that there is nothing in the db\r
+\r
+ * add a message to it\r
+\r
+ * run "notmuch new"\r
+\r
+ * use delve -a (from xapian-tools) to see the list of terms\r
+\r
+ * remove the message\r
+\r
+ * run "notmuch new"\r
+\r
+ * use delve -a to see that the ghost terms remain\r
+\r
+\r
+This suggests that if i include a bogus message-id in my References:\r
+list for any mail i send to a notmuch user, then access to their\r
+database will let me tell whether they ever saw it or not, regardless of\r
+whether they deleted the message.\r
+\r
+Deletion by the user should be final -- i don't like it when tools\r
+snitch on their users.\r
+\r
+How should this be fixed?\r
+\r
+     --dkg\r