Re: [notmuch] 25 minutes load time with emacs -f notmuch

diff --git a/3d/f412ca63694f824d8e7a137ced02d78000cb21 b/3d/f412ca63694f824d8e7a137ced02d78000cb21
new file mode 100644 (file)
index 0000000..bc5f2e5
--- /dev/null
+++ b/3d/f412ca63694f824d8e7a137ced02d78000cb21
@@ -0,0 +1,74 @@
+Return-Path: <mh@glandium.org>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+       by olra.theworths.org (Postfix) with ESMTP id 48066431FBC\r
+       for <notmuch@notmuchmail.org>; Sun, 22 Nov 2009 00:37:31 -0800 (PST)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+       by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+       with ESMTP id 07CYxhR9p5Wi for <notmuch@notmuchmail.org>;\r
+       Sun, 22 Nov 2009 00:37:30 -0800 (PST)\r
+Received: from vuizook.err.no (vuizook.err.no [85.19.221.46])\r
+       by olra.theworths.org (Postfix) with ESMTP id 51C1C431FAE\r
+       for <notmuch@notmuchmail.org>; Sun, 22 Nov 2009 00:37:30 -0800 (PST)\r
+Received: from cha92-13-88-165-248-19.fbx.proxad.net ([88.165.248.19]\r
+       helo=jigen)\r
+       by vuizook.err.no with esmtps (TLS-1.0:RSA_AES_256_CBC_SHA1:32)\r
+       (Exim 4.69) (envelope-from <mh@glandium.org>)\r
+       id 1NC7wk-0006Ts-2P; Sun, 22 Nov 2009 09:37:28 +0100\r
+Received: from mh by jigen with local (Exim 4.69) (envelope-from <mh@jigen>)\r
+       id 1NC7w6-0002OZ-93; Sun, 22 Nov 2009 09:36:46 +0100\r
+Date: Sun, 22 Nov 2009 09:36:46 +0100\r
+From: Mike Hommey <mh+notmuch@glandium.org>\r
+To: Brett Viren <brett.viren@gmail.com>\r
+Message-ID: <20091122083646.GA8420@glandium.org>\r
+References: <20091121145111.GB19397@excalibur.local>\r
+       <87fx874xj5.fsf@yoom.home.cworth.org>\r
+       <46263c600911211436s5826015eqc5fc18a4164245cb@mail.gmail.com>\r
+MIME-Version: 1.0\r
+Content-Type: text/plain; charset=us-ascii\r
+Content-Disposition: inline\r
+In-Reply-To: <46263c600911211436s5826015eqc5fc18a4164245cb@mail.gmail.com>\r
+X-GPG-Fingerprint: A479 A824 265C B2A5 FC54  8D1E DE4B DA2C 54FD 2A58\r
+User-Agent: Mutt/1.5.20 (2009-06-14)\r
+Cc: notmuch@notmuchmail.org\r
+Subject: Re: [notmuch] 25 minutes load time with emacs -f notmuch\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.12\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+       <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Sun, 22 Nov 2009 08:37:31 -0000\r
+\r
+On Sat, Nov 21, 2009 at 05:36:18PM -0500, Brett Viren wrote:\r
+> On Sat, Nov 21, 2009 at 12:07 PM, Carl Worth <cworth@cworth.org> wrote:\r
+> \r
+> > Though, frankly, I think we need to fix "notmuch new" to do much better\r
+> > than 40 files/sec.\r
+> \r
+> Just a "me too".\r
+> \r
+> Processed 130871 total files in 38m 7s (57 files/sec.).\r
+> Added 102723 new messages to the database (not much, really).\r
+> \r
+> This was ~2GB of mail on a 2.5GHz CPU.  That seems pretty reasonable\r
+> to me but I'd like to rerun the "notmuch new" under google perftools\r
+> to see if there are any obvious bottlenecks that might be cleaned up.\r
+\r
+FWIW, my 90k+ messages mailbox was imported at a pace of 130 files/sec,\r
+and my CPU is "only" 2.2GHz, but I have a SSD. A good share of the\r
+bottlenecks is "simply" I/O. Don't forget having a lot of small files\r
+sucks I/O wise, as files are most likely spread all over the disk.\r
+\r
+A good test, if you have enough memory, would be to put your mailbox in\r
+a tmpfs, and see how fast that imports.\r
+\r
+Mike\r