Re: [notmuch] Notmuch performance (literally, in my case)

diff --git a/58/9d9e61ceffca58b35400bbe64fa2a8d456f854 b/58/9d9e61ceffca58b35400bbe64fa2a8d456f854
new file mode 100644 (file)
index 0000000..c375a2f
--- /dev/null
+++ b/58/9d9e61ceffca58b35400bbe64fa2a8d456f854
@@ -0,0 +1,91 @@
+Return-Path: <aneesh.kumar@linux.vnet.ibm.com>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+       by olra.theworths.org (Postfix) with ESMTP id 51075404947\r
+       for <notmuch@notmuchmail.org>; Tue, 16 Mar 2010 10:10:28 -0700 (PDT)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -3.943\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-3.943 tagged_above=-999 required=5 tests=[AWL=0.056,\r
+       BAYES_50=0.001, RCVD_IN_DNSWL_MED=-4] autolearn=ham\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+       by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+       with ESMTP id XdhZlTgZUa5F for <notmuch@notmuchmail.org>;\r
+       Tue, 16 Mar 2010 10:10:25 -0700 (PDT)\r
+Received: from e23smtp02.au.ibm.com (e23smtp02.au.ibm.com [202.81.31.144])\r
+       by olra.theworths.org (Postfix) with ESMTP id EE385404942\r
+       for <notmuch@notmuchmail.org>; Tue, 16 Mar 2010 10:10:24 -0700 (PDT)\r
+Received: from d23relay05.au.ibm.com (d23relay05.au.ibm.com [202.81.31.247])\r
+       by e23smtp02.au.ibm.com (8.14.3/8.13.1) with ESMTP id o2GH77Yg012514\r
+       for <notmuch@notmuchmail.org>; Wed, 17 Mar 2010 04:07:07 +1100\r
+Received: from d23av03.au.ibm.com (d23av03.au.ibm.com [9.190.234.97])\r
+       by d23relay05.au.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id\r
+       o2GH4S0q1454158\r
+       for <notmuch@notmuchmail.org>; Wed, 17 Mar 2010 04:04:28 +1100\r
+Received: from d23av03.au.ibm.com (loopback [127.0.0.1])\r
+       by d23av03.au.ibm.com (8.14.3/8.13.1/NCO v10.0 AVout) with ESMTP id\r
+       o2GHAMKQ027341\r
+       for <notmuch@notmuchmail.org>; Wed, 17 Mar 2010 04:10:22 +1100\r
+Received: from skywalker.linux.vnet.ibm.com ([9.77.214.188])\r
+       by d23av03.au.ibm.com (8.14.3/8.13.1/NCO v10.0 AVin) with ESMTP id\r
+       o2GHAHqS027185; Wed, 17 Mar 2010 04:10:19 +1100\r
+From: "Aneesh Kumar K. V" <aneesh.kumar@linux.vnet.ibm.com>\r
+To: Ben Gamari <bgamari.foss@gmail.com>, Olly Betts <olly@survex.com>,\r
+       martin f krafft <madduck@madduck.net>\r
+In-Reply-To: <4b9fa5d2.0a4d5e0a.0c0b.ffffdcbb@mx.google.com>\r
+References: <4b9dccc0.c6c1f10a.3671.44ec@mx.google.com>\r
+       <20100315090401.GA29891@glaive.weftsoar.net>\r
+       <slrnhprvfv.hu6.olly@msgid.survex.com>\r
+       <4b9e6e80.09b6660a.6769.6832@mx.google.com>\r
+       <20100316110846.GK10323@survex.com>\r
+       <4b9fa5d2.0a4d5e0a.0c0b.ffffdcbb@mx.google.com>\r
+Date: Tue, 16 Mar 2010 22:40:17 +0530\r
+Message-ID: <87mxy8chvq.fsf@linux.vnet.ibm.com>\r
+MIME-Version: 1.0\r
+Content-Type: text/plain; charset=us-ascii\r
+Cc: notmuch@notmuchmail.org\r
+Subject: Re: [notmuch] Notmuch performance (literally, in my case)\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+       <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Tue, 16 Mar 2010 17:10:28 -0000\r
+\r
+On Tue, 16 Mar 2010 08:37:54 -0700 (PDT), Ben Gamari <bgamari.foss@gmail.com> wrote:\r
+> On Tue, 16 Mar 2010 11:08:47 +0000, Olly Betts <olly@survex.com> wrote:\r
+> > For the issue of a background task interfering with interactive use, the feel\r
+> > arguably matters more than the throughput.\r
+> > \r
+> > I'll probably put that patch in 1.0.19, and look at moving all the fdatasync()\r
+> > calls together.  This is http://trac.xapian.org/ticket/426 BTW.\r
+> > \r
+> > The kernel should be able to handle this workload better though, so I would\r
+> > say it was worthwhile to bring up on LKML if you have the energy.  It certainly\r
+> > isn't just you, as apt-xapian-index seems to trigger it for some Ubuntu users,\r
+> > and madduck mentioned it on #notmuch a week or so ago.\r
+> \r
+> Alright. This issue has been bothering me for a very long time and it's frankly\r
+> pretty pathetic how badly the kernel falls apart under this sort of workload.\r
+> I just wrote up a message (4b9fa440.12135e0a.7fc8.ffffe745@mx.google.com), so\r
+> we'll see what happens. In the past kernel developers have been very eager to\r
+> write this issue off as not reproducible enough (perhaps wisely), so if anyone\r
+> has anything to say, please contribute it to the thread.\r
+> \r
+\r
+Ext3 fsync related issue is a know problem due to the way journalling is\r
+handled in ext3. The solution for that would be data=writeback ( with\r
+its loss of data integrity ) or not yet upstreamed data=guarded. Another\r
+option would be to try ext4 which should not be impacted that badly by\r
+the data=ordered journalled mode\r
+\r
+-aneesh\r