From: David Bremner <david@tethera.net>
To: Nils Dagsson Moskopp <nils@dieweltistgarnichtso.net>,
	notmuch@notmuchmail.org
Subject: Re: notmuch-hello buffer slow doe to slow query
In-Reply-To: <874n20go84.fsf@dieweltistgarnichtso.net>
References: <874n20go84.fsf@dieweltistgarnichtso.net>
User-Agent: Notmuch/0.17+180~g8977b1a (http://notmuchmail.org) Emacs/24.3.1
	(x86_64-pc-linux-gnu)
Date: Sat, 12 Apr 2014 12:41:11 -0300
Message-ID: <87ppkm4ktk.fsf@zancas.localnet>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
Precedence: list

--=-=-=
Content-Type: text/plain

Nils Dagsson Moskopp <nils@dieweltistgarnichtso.net> writes:

>
> mjw1009 can reproduce if the from-query contains an "@" and thinks the
> problem may be "something deeper down in notmuch (actually probably in
> xapian)".

My own experiments contradict that (see attached patch for a test to
run). The exact address is about twice as fast to count on both of my
machines, which matches the amount of I/O. FWIW, my SSD based laptop is
about 10x faster (compared to a similarly specced desktop with HD) for
both of them, which kindof reinforces how I/O bound this is.  

Replacing 'count' with 'search' gives pretty similar relative behaviour,
although much slower since it seems to do about 10x I/O.  

Finally after an initial "notmuch tag +fromlinus
from:torvalds@linux-foundation.org", "notmuch count tag:fromlinus" seems
to be about 5-10x faster than "notmuch count
from:torvalds@linux-foundation.org".  So that might be a workaround for
some people.

To sum up, I think the underlying question is whether the amount of I/O
that count does can be reduced when using non-boolean query terms.


--=-=-=
Content-Type: text/x-diff
Content-Disposition: inline;
 filename=0001-WIP-new-performance-test-for-notmuch-count.patch

>From d3fbbe7ac0a1fb5b5691674226a3a3d000e15a1c Mon Sep 17 00:00:00 2001
From: David Bremner <david@tethera.net>
Date: Sat, 12 Apr 2014 11:50:22 -0300
Subject: [PATCH] WIP: new performance test for notmuch count

Probably not suitable for production because of sudo
---
 performance-test/T03-count.sh | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)
 create mode 100755 performance-test/T03-count.sh

diff --git a/performance-test/T03-count.sh b/performance-test/T03-count.sh
new file mode 100755
index 0000000..e74437e
--- /dev/null
+++ b/performance-test/T03-count.sh
@@ -0,0 +1,23 @@
+#!/bin/bash
+
+test_description='counting'
+
+. ./perf-test-lib.sh
+
+time_start
+
+for i in $(seq 1 10); do
+    echo 3 | sudo tee /proc/sys/vm/drop_caches
+    notmuch --version > /dev/null
+    time_run "from:torvalds@linux-foundation.org  " \
+	"notmuch count from:torvalds@linux-foundation.org"
+
+    echo 3 | sudo tee /proc/sys/vm/drop_caches
+    notmuch --version > /dev/null
+
+    time_run "from:linux-foundation.org           " \
+	"notmuch count from:linux-foundation.org"
+done
+
+
+time_done
-- 
1.9.1


--=-=-=--