Re: notmuch-hello buffer slow doe to slow query

diff --git a/38/3d59113a084557d8ec72bd84edbd71c17285f9 b/38/3d59113a084557d8ec72bd84edbd71c17285f9
new file mode 100644 (file)
index 0000000..bba9575
--- /dev/null
+++ b/38/3d59113a084557d8ec72bd84edbd71c17285f9
@@ -0,0 +1,131 @@
+Return-Path: <david@tethera.net>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+       by olra.theworths.org (Postfix) with ESMTP id AE001431FBD\r
+       for <notmuch@notmuchmail.org>; Sat, 12 Apr 2014 08:41:27 -0700 (PDT)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: 0\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=0 tagged_above=-999 required=5 tests=[none]\r
+       autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+       by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+       with ESMTP id a0YmBTZ+V4d5 for <notmuch@notmuchmail.org>;\r
+       Sat, 12 Apr 2014 08:41:21 -0700 (PDT)\r
+Received: from yantan.tethera.net (yantan.tethera.net [199.188.72.155])\r
+       (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits))\r
+       (No client certificate requested)\r
+       by olra.theworths.org (Postfix) with ESMTPS id 43C76431FBC\r
+       for <notmuch@notmuchmail.org>; Sat, 12 Apr 2014 08:41:21 -0700 (PDT)\r
+Received: from remotemail by yantan.tethera.net with local (Exim 4.80)\r
+       (envelope-from <david@tethera.net>)\r
+       id 1WZ03P-00044j-Tg; Sat, 12 Apr 2014 12:41:15 -0300\r
+Received: (nullmailer pid 12090 invoked by uid 1000); Sat, 12 Apr 2014\r
+       15:41:11 -0000\r
+From: David Bremner <david@tethera.net>\r
+To: Nils Dagsson Moskopp <nils@dieweltistgarnichtso.net>,\r
+       notmuch@notmuchmail.org\r
+Subject: Re: notmuch-hello buffer slow doe to slow query\r
+In-Reply-To: <874n20go84.fsf@dieweltistgarnichtso.net>\r
+References: <874n20go84.fsf@dieweltistgarnichtso.net>\r
+User-Agent: Notmuch/0.17+180~g8977b1a (http://notmuchmail.org) Emacs/24.3.1\r
+       (x86_64-pc-linux-gnu)\r
+Date: Sat, 12 Apr 2014 12:41:11 -0300\r
+Message-ID: <87ppkm4ktk.fsf@zancas.localnet>\r
+MIME-Version: 1.0\r
+Content-Type: multipart/mixed; boundary="=-=-="\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+       <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Sat, 12 Apr 2014 15:41:27 -0000\r
+\r
+--=-=-=\r
+Content-Type: text/plain\r
+\r
+Nils Dagsson Moskopp <nils@dieweltistgarnichtso.net> writes:\r
+\r
+>\r
+> mjw1009 can reproduce if the from-query contains an "@" and thinks the\r
+> problem may be "something deeper down in notmuch (actually probably in\r
+> xapian)".\r
+\r
+My own experiments contradict that (see attached patch for a test to\r
+run). The exact address is about twice as fast to count on both of my\r
+machines, which matches the amount of I/O. FWIW, my SSD based laptop is\r
+about 10x faster (compared to a similarly specced desktop with HD) for\r
+both of them, which kindof reinforces how I/O bound this is.  \r
+\r
+Replacing 'count' with 'search' gives pretty similar relative behaviour,\r
+although much slower since it seems to do about 10x I/O.  \r
+\r
+Finally after an initial "notmuch tag +fromlinus\r
+from:torvalds@linux-foundation.org", "notmuch count tag:fromlinus" seems\r
+to be about 5-10x faster than "notmuch count\r
+from:torvalds@linux-foundation.org".  So that might be a workaround for\r
+some people.\r
+\r
+To sum up, I think the underlying question is whether the amount of I/O\r
+that count does can be reduced when using non-boolean query terms.\r
+\r
+\r
+--=-=-=\r
+Content-Type: text/x-diff\r
+Content-Disposition: inline;\r
+ filename=0001-WIP-new-performance-test-for-notmuch-count.patch\r
+\r
+>From d3fbbe7ac0a1fb5b5691674226a3a3d000e15a1c Mon Sep 17 00:00:00 2001\r
+From: David Bremner <david@tethera.net>\r
+Date: Sat, 12 Apr 2014 11:50:22 -0300\r
+Subject: [PATCH] WIP: new performance test for notmuch count\r
+\r
+Probably not suitable for production because of sudo\r
+---\r
+ performance-test/T03-count.sh | 23 +++++++++++++++++++++++\r
+ 1 file changed, 23 insertions(+)\r
+ create mode 100755 performance-test/T03-count.sh\r
+\r
+diff --git a/performance-test/T03-count.sh b/performance-test/T03-count.sh\r
+new file mode 100755\r
+index 0000000..e74437e\r
+--- /dev/null\r
++++ b/performance-test/T03-count.sh\r
+@@ -0,0 +1,23 @@\r
++#!/bin/bash\r
++\r
++test_description='counting'\r
++\r
++. ./perf-test-lib.sh\r
++\r
++time_start\r
++\r
++for i in $(seq 1 10); do\r
++    echo 3 | sudo tee /proc/sys/vm/drop_caches\r
++    notmuch --version > /dev/null\r
++    time_run "from:torvalds@linux-foundation.org  " \\r
++      "notmuch count from:torvalds@linux-foundation.org"\r
++\r
++    echo 3 | sudo tee /proc/sys/vm/drop_caches\r
++    notmuch --version > /dev/null\r
++\r
++    time_run "from:linux-foundation.org           " \\r
++      "notmuch count from:linux-foundation.org"\r
++done\r
++\r
++\r
++time_done\r
+-- \r
+1.9.1\r
+\r
+\r
+--=-=-=--\r