From: Austin Clements Date: Sat, 4 Oct 2014 20:15:13 +0000 (+2000) Subject: Re: [PATCH 00/11] Add ghost messages and fix thread linking X-Git-Url: http://git.tremily.us/?a=commitdiff_plain;h=99ea16cc1a2863f06869ade2164fea92af69252c;p=notmuch-archives.git Re: [PATCH 00/11] Add ghost messages and fix thread linking --- diff --git a/f5/3205bfe73f9b8e54379d6e2b218b1ecf8a3ddb b/f5/3205bfe73f9b8e54379d6e2b218b1ecf8a3ddb new file mode 100644 index 000000000..06b57837f --- /dev/null +++ b/f5/3205bfe73f9b8e54379d6e2b218b1ecf8a3ddb @@ -0,0 +1,103 @@ +Return-Path: +X-Original-To: notmuch@notmuchmail.org +Delivered-To: notmuch@notmuchmail.org +Received: from localhost (localhost [127.0.0.1]) + by olra.theworths.org (Postfix) with ESMTP id 4EF98431FBC + for ; Sat, 4 Oct 2014 13:15:26 -0700 (PDT) +X-Virus-Scanned: Debian amavisd-new at olra.theworths.org +X-Spam-Flag: NO +X-Spam-Score: -2.3 +X-Spam-Level: +X-Spam-Status: No, score=-2.3 tagged_above=-999 required=5 + tests=[RCVD_IN_DNSWL_MED=-2.3] autolearn=disabled +Received: from olra.theworths.org ([127.0.0.1]) + by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) + with ESMTP id NERxwWXStbjr for ; + Sat, 4 Oct 2014 13:15:19 -0700 (PDT) +Received: from outgoing.csail.mit.edu (outgoing.csail.mit.edu [128.30.2.149]) + by olra.theworths.org (Postfix) with ESMTP id 78308431FB6 + for ; Sat, 4 Oct 2014 13:15:19 -0700 (PDT) +Received: from [104.131.20.129] (helo=awakeningjr) + by outgoing.csail.mit.edu with esmtpsa (TLS1.0:RSA_AES_128_CBC_SHA1:16) + (Exim 4.72) (envelope-from ) + id 1XaVjW-0006BA-Af; Sat, 04 Oct 2014 16:15:14 -0400 +Received: from amthrax by awakeningjr with local (Exim 4.84) + (envelope-from ) + id 1XaVjV-00054i-If; Sat, 04 Oct 2014 16:15:13 -0400 +From: Austin Clements +To: Tomi Ollila , notmuch@notmuchmail.org +Subject: Re: [PATCH 00/11] Add ghost messages and fix thread linking +In-Reply-To: +References: <1412345958-8278-1-git-send-email-aclements@csail.mit.edu> + +User-Agent: Notmuch/0.18.1+86~gef5e66a (http://notmuchmail.org) Emacs/24.3.1 + (x86_64-pc-linux-gnu) +Date: Sat, 04 Oct 2014 16:15:13 -0400 +Message-ID: <87ppe7d226.fsf@csail.mit.edu> +MIME-Version: 1.0 +Content-Type: text/plain +X-BeenThere: notmuch@notmuchmail.org +X-Mailman-Version: 2.1.13 +Precedence: list +List-Id: "Use and development of the notmuch mail system." + +List-Unsubscribe: , + +List-Archive: +List-Post: +List-Help: +List-Subscribe: , + +X-List-Received-Date: Sat, 04 Oct 2014 20:15:26 -0000 + +On Sat, 04 Oct 2014, Tomi Ollila wrote: +> On Fri, Oct 03 2014, Austin Clements wrote: +> +>> This series modifies our database representation of messages that have +>> been referenced by other messages, but for which we don't have the +>> message itself. Currently, we store this information as Xapian +>> metadata, but this has several downsides for performance and +>> complexity and results in hard-to-fix thread linking bugs. This patch +>> series implements "ghost messages", which replace this Xapian metadata +>> with Xapian documents that look and act very much like regular message +>> documents, but simply have no content. This simplifies and speeds up +>> our thread linking algorithm and fixes the currently broken thread +>> linking test. +>> +>> Ghost messages also open up interesting future possibilities, such as +>> "pre-seeding" tags for messages that are not yet indexed. This could +>> be used to make notmuch insert simpler and more robust, as part of tag +>> synchronization, and to improve nmbug's behavior when tags arrive +>> before messages. +> +> The code looks OK to me -- there are IMO some strange things but those +> don't change the status quo -- I did look a little past the diffs into +> the code to understand something... +> +> I am now having these patches applied to my 'home' notmuch and haven't yet +> seen anything strange there. In this setup I have 27 emails missing that +> nmbug expects there to be -- let's see if I can get ghost messages there. + +In its current form, it won't help with these missing messages from +nmbug. In the future, nmbug could use ghost messages to record the tags +for those messages anyway and then it becomes an interface question of +whether nmbug should report such missing messages. + +> ... and tests pass, ran while writing the above part... +> +> I'll put these in use next week on one of my 'work' notmuch. There I have +> seen a problem that while I am on one thread, +> `notmuch-poll-and-refresh-buffer' picks new mail but suddenly this thread +> now has changed it's thread id :O -- making the search buffer go blank. +> I'll see whether it still happens with these (may be totally unrelated) +> and perhaps investigate further... + +I have this same problem with some people I communicate with. I haven't +looked in to it, but I assume their messages reference only the +immediate parent, so if it's a reply to another unindexed message then +the thread ID may change depending on indexing order and other arbitrary +factors. I may be wrong, but if this is the case, I don't think ghost +messages will change this behavior. Solving this may require more +intelligently picking the "winning" thread ID when merging threads; +e.g., pick the thread ID with the most messages, rather than picking +arbitrarily.