Return-Path: <amthrax@awakening.csail.mit.edu>
X-Original-To: notmuch@notmuchmail.org
Delivered-To: notmuch@notmuchmail.org
Received: from localhost (localhost [127.0.0.1])
	by olra.theworths.org (Postfix) with ESMTP id 5DB21431FB6
	for <notmuch@notmuchmail.org>; Thu,  9 Dec 2010 13:05:06 -0800 (PST)
X-Virus-Scanned: Debian amavisd-new at olra.theworths.org
X-Spam-Flag: NO
X-Spam-Score: 0
X-Spam-Level: 
X-Spam-Status: No, score=0 tagged_above=-999 required=5 tests=[none]
	autolearn=disabled
Received: from olra.theworths.org ([127.0.0.1])
	by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)
	with ESMTP id GmHR63m4ML7z for <notmuch@notmuchmail.org>;
	Thu,  9 Dec 2010 13:05:06 -0800 (PST)
X-Greylist: delayed 300 seconds by postgrey-1.32 at olra;
	Thu, 09 Dec 2010 13:05:05 PST
Received: from dmz-mailsec-scanner-4.mit.edu (DMZ-MAILSEC-SCANNER-4.MIT.EDU
	[18.9.25.15])
	by olra.theworths.org (Postfix) with ESMTP id EB470431FB5
	for <notmuch@notmuchmail.org>; Thu,  9 Dec 2010 13:05:05 -0800 (PST)
X-AuditID: 1209190f-b7c1dae000000a2b-02-4d0143551157
Received: from mailhub-auth-3.mit.edu ( [18.9.21.43])
	by dmz-mailsec-scanner-4.mit.edu (Symantec Brightmail Gateway) with
	SMTP id 84.E2.02603.553410D4; Thu,  9 Dec 2010 16:00:05 -0500 (EST)
Received: from outgoing.mit.edu (OUTGOING-AUTH.MIT.EDU [18.7.22.103])
	by mailhub-auth-3.mit.edu (8.13.8/8.9.2) with ESMTP id oB9L04fO026394
	for <notmuch@notmuchmail.org>; Thu, 9 Dec 2010 16:00:05 -0500
Received: from awakening.csail.mit.edu (awakening.csail.mit.edu [18.26.4.91])
	(authenticated bits=0)
	(User authenticated as amdragon@ATHENA.MIT.EDU)
	by outgoing.mit.edu (8.13.6/8.12.4) with ESMTP id oB9L01RF009974
	(version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NOT)
	for <notmuch@notmuchmail.org>; Thu, 9 Dec 2010 16:00:04 -0500 (EST)
Received: from amthrax by awakening.csail.mit.edu with local (Exim 4.72)
	(envelope-from <amthrax@awakening.csail.mit.edu>) id 1PQnar-0007Hg-S9
	for notmuch@notmuchmail.org; Thu, 09 Dec 2010 16:00:01 -0500
From: Austin Clements <amdragon@MIT.EDU>
To: notmuch@notmuchmail.org
Subject: [PATCH 0/5] Fetch all message metadata in a single pass
Date: Thu,  9 Dec 2010 15:59:51 -0500
Message-Id: <1291928396-27937-1-git-send-email-amdragon@mit.edu>
X-Mailer: git-send-email 1.7.2.3
X-Brightmail-Tracker: AAAAAA==
X-BeenThere: notmuch@notmuchmail.org
X-Mailman-Version: 2.1.13
Precedence: list
List-Id: "Use and development of the notmuch mail system."
	<notmuch.notmuchmail.org>
List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,
	<mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>
List-Archive: <http://notmuchmail.org/pipermail/notmuch>
List-Post: <mailto:notmuch@notmuchmail.org>
List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>
List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,
	<mailto:notmuch-request@notmuchmail.org?subject=subscribe>
X-List-Received-Date: Thu, 09 Dec 2010 21:05:06 -0000

This is the second of the two optimizations I described a while ago,
and brings my inbox search down to 1.811 seconds, 2.5X faster than it
was originally.

This optimization is based on the observation that Xapian decompresses
a document's term list every time you iterate over it.  As a result,
notmuch can decompress the beginning of a single term list quite a few
times.  This patch series combines all of this into a single pass that
is only slightly more expensive than fetching one metadata field used
to be, but offers a huge win in the common case where a message object
is used for multiple metadata fields.