Return-Path: <james@jameswestby.net>
X-Original-To: notmuch@notmuchmail.org
Delivered-To: notmuch@notmuchmail.org
Received: from localhost (localhost [127.0.0.1])
	by olra.theworths.org (Postfix) with ESMTP id 2FFC2431FAE
	for <notmuch@notmuchmail.org>; Fri, 18 Dec 2009 11:02:31 -0800 (PST)
X-Virus-Scanned: Debian amavisd-new at olra.theworths.org
Received: from olra.theworths.org ([127.0.0.1])
	by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)
	with ESMTP id EPAqHHbqLDbF for <notmuch@notmuchmail.org>;
	Fri, 18 Dec 2009 11:02:30 -0800 (PST)
Received: from mout.perfora.net (mout.perfora.net [74.208.4.195])
	by olra.theworths.org (Postfix) with ESMTP id 6C5F8431FC3
	for <notmuch@notmuchmail.org>; Fri, 18 Dec 2009 11:02:30 -0800 (PST)
Received: from jameswestby.net (jameswestby.net [89.145.97.141])
	by mx.perfora.net (node=mxus2) with ESMTP (Nemesis)
	id 0MA8Qr-1NAThE1ykT-00BOWU for notmuch@notmuchmail.org;
	Fri, 18 Dec 2009 14:02:29 -0500
Received: from cpc4-aztw22-2-0-cust59.aztw.cable.virginmedia.com
	([94.169.116.60] helo=flash)
	by jameswestby.net with esmtpa (Exim 4.69)
	(envelope-from <james@jameswestby.net>) id 1NLi5r-0005oR-5z
	for notmuch@notmuchmail.org; Fri, 18 Dec 2009 19:02:27 +0000
Received: by flash (Postfix, from userid 1000)
	id 90CFA6E546A; Fri, 18 Dec 2009 19:02:21 +0000 (GMT)
From: James Westby <jw+debian@jameswestby.net>
To: notmuch@notmuchmail.org
Date: Fri, 18 Dec 2009 19:02:21 +0000
Message-ID: <87oclwrtqa.fsf@jameswestby.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Subject: [notmuch] Missing messages breaking threads
X-BeenThere: notmuch@notmuchmail.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Use and development of the notmuch mail system."
	<notmuch.notmuchmail.org>
List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,
	<mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>
List-Archive: <http://notmuchmail.org/pipermail/notmuch>
List-Post: <mailto:notmuch@notmuchmail.org>
List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>
List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,
	<mailto:notmuch-request@notmuchmail.org?subject=subscribe>
X-List-Received-Date: Fri, 18 Dec 2009 19:02:31 -0000

Hi,

I like the architecture of notmuch, and have just switched
to using it as my primary client, so thanks.

I however have discovered one issue that is a pain.

I use a bugtracker a lot which has workflows that mean that
you don't always get teh initial messages from a bug.

To put it in more common terms, imagine this:

  * Alice sends a mail to a large group of your friends, but
    not you.
  * Each of these friends replies, and puts you in Cc for the
    reply.

This will mean that you get several messages that all have
References and In-Reply-To set to ids that aren't known to
notmuch.

This means that it doesn't thread them, and so they aren't
grouped in the UI. This becomes painful when you are dealing
with several bugs like this. It's almost like being back in
the days of a message oriented client, and we know that's
not the way to do it.

Therefore I'd like to fix this. The obvious way is to
introduce documents in to the db for each id we see, and
threading should then naturally work better.

The only issue I see with doing this is with mail delays.
Once we do this we will sometimes receive a message that
already has a dummy document. What happens currently with
message-id collisions?

Therefore I would propose this:

  * When doing a thread resolution and we have ids that
    we don't know, add a document to the db that is
    something like

    {id: <message-id>
     thread: <synthesised thread-id>
     dummy: True
     content: "Messages missing"
    }

  * When we get a message-id conflict check for dummy:True
    and replace the document if it is there.

How does this sound?

There could be an issue with synthesising too many threads
and then ending up having to try and put a message in two
threads? I see there is code for merging threads, would that
handle this?

Thanks,

James