Date: Sun, 22 Nov 2009 09:36:46 +0100
From: Mike Hommey <mh+notmuch@glandium.org>
To: Brett Viren <brett.viren@gmail.com>
Message-ID: <20091122083646.GA8420@glandium.org>
References: <20091121145111.GB19397@excalibur.local>
	<87fx874xj5.fsf@yoom.home.cworth.org>
	<46263c600911211436s5826015eqc5fc18a4164245cb@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <46263c600911211436s5826015eqc5fc18a4164245cb@mail.gmail.com>
User-Agent: Mutt/1.5.20 (2009-06-14)
Cc: notmuch@notmuchmail.org
Subject: Re: [notmuch] 25 minutes load time with emacs -f notmuch
Precedence: list

On Sat, Nov 21, 2009 at 05:36:18PM -0500, Brett Viren wrote:
> On Sat, Nov 21, 2009 at 12:07 PM, Carl Worth <cworth@cworth.org> wrote:
> 
> > Though, frankly, I think we need to fix "notmuch new" to do much better
> > than 40 files/sec.
> 
> Just a "me too".
> 
> Processed 130871 total files in 38m 7s (57 files/sec.).
> Added 102723 new messages to the database (not much, really).
> 
> This was ~2GB of mail on a 2.5GHz CPU.  That seems pretty reasonable
> to me but I'd like to rerun the "notmuch new" under google perftools
> to see if there are any obvious bottlenecks that might be cleaned up.

FWIW, my 90k+ messages mailbox was imported at a pace of 130 files/sec,
and my CPU is "only" 2.2GHz, but I have a SSD. A good share of the
bottlenecks is "simply" I/O. Don't forget having a lot of small files
sucks I/O wise, as files are most likely spread all over the disk.

A good test, if you have enough memory, would be to put your mailbox in
a tmpfs, and see how fast that imports.

Mike