From fbe7f1b65ef49d3ee1d6d2d000fcf4183f4972e1 Mon Sep 17 00:00:00 2001 From: David Bremner Date: Sun, 13 Apr 2014 12:41:11 +2100 Subject: [PATCH] Re: notmuch-hello buffer slow doe to slow query --- 38/3d59113a084557d8ec72bd84edbd71c17285f9 | 131 ++++++++++++++++++++++ 1 file changed, 131 insertions(+) create mode 100644 38/3d59113a084557d8ec72bd84edbd71c17285f9 diff --git a/38/3d59113a084557d8ec72bd84edbd71c17285f9 b/38/3d59113a084557d8ec72bd84edbd71c17285f9 new file mode 100644 index 000000000..bba95758e --- /dev/null +++ b/38/3d59113a084557d8ec72bd84edbd71c17285f9 @@ -0,0 +1,131 @@ +Return-Path: +X-Original-To: notmuch@notmuchmail.org +Delivered-To: notmuch@notmuchmail.org +Received: from localhost (localhost [127.0.0.1]) + by olra.theworths.org (Postfix) with ESMTP id AE001431FBD + for ; Sat, 12 Apr 2014 08:41:27 -0700 (PDT) +X-Virus-Scanned: Debian amavisd-new at olra.theworths.org +X-Spam-Flag: NO +X-Spam-Score: 0 +X-Spam-Level: +X-Spam-Status: No, score=0 tagged_above=-999 required=5 tests=[none] + autolearn=disabled +Received: from olra.theworths.org ([127.0.0.1]) + by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) + with ESMTP id a0YmBTZ+V4d5 for ; + Sat, 12 Apr 2014 08:41:21 -0700 (PDT) +Received: from yantan.tethera.net (yantan.tethera.net [199.188.72.155]) + (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) + (No client certificate requested) + by olra.theworths.org (Postfix) with ESMTPS id 43C76431FBC + for ; Sat, 12 Apr 2014 08:41:21 -0700 (PDT) +Received: from remotemail by yantan.tethera.net with local (Exim 4.80) + (envelope-from ) + id 1WZ03P-00044j-Tg; Sat, 12 Apr 2014 12:41:15 -0300 +Received: (nullmailer pid 12090 invoked by uid 1000); Sat, 12 Apr 2014 + 15:41:11 -0000 +From: David Bremner +To: Nils Dagsson Moskopp , + notmuch@notmuchmail.org +Subject: Re: notmuch-hello buffer slow doe to slow query +In-Reply-To: <874n20go84.fsf@dieweltistgarnichtso.net> +References: <874n20go84.fsf@dieweltistgarnichtso.net> +User-Agent: Notmuch/0.17+180~g8977b1a (http://notmuchmail.org) Emacs/24.3.1 + (x86_64-pc-linux-gnu) +Date: Sat, 12 Apr 2014 12:41:11 -0300 +Message-ID: <87ppkm4ktk.fsf@zancas.localnet> +MIME-Version: 1.0 +Content-Type: multipart/mixed; boundary="=-=-=" +X-BeenThere: notmuch@notmuchmail.org +X-Mailman-Version: 2.1.13 +Precedence: list +List-Id: "Use and development of the notmuch mail system." + +List-Unsubscribe: , + +List-Archive: +List-Post: +List-Help: +List-Subscribe: , + +X-List-Received-Date: Sat, 12 Apr 2014 15:41:27 -0000 + +--=-=-= +Content-Type: text/plain + +Nils Dagsson Moskopp writes: + +> +> mjw1009 can reproduce if the from-query contains an "@" and thinks the +> problem may be "something deeper down in notmuch (actually probably in +> xapian)". + +My own experiments contradict that (see attached patch for a test to +run). The exact address is about twice as fast to count on both of my +machines, which matches the amount of I/O. FWIW, my SSD based laptop is +about 10x faster (compared to a similarly specced desktop with HD) for +both of them, which kindof reinforces how I/O bound this is. + +Replacing 'count' with 'search' gives pretty similar relative behaviour, +although much slower since it seems to do about 10x I/O. + +Finally after an initial "notmuch tag +fromlinus +from:torvalds@linux-foundation.org", "notmuch count tag:fromlinus" seems +to be about 5-10x faster than "notmuch count +from:torvalds@linux-foundation.org". So that might be a workaround for +some people. + +To sum up, I think the underlying question is whether the amount of I/O +that count does can be reduced when using non-boolean query terms. + + +--=-=-= +Content-Type: text/x-diff +Content-Disposition: inline; + filename=0001-WIP-new-performance-test-for-notmuch-count.patch + +>From d3fbbe7ac0a1fb5b5691674226a3a3d000e15a1c Mon Sep 17 00:00:00 2001 +From: David Bremner +Date: Sat, 12 Apr 2014 11:50:22 -0300 +Subject: [PATCH] WIP: new performance test for notmuch count + +Probably not suitable for production because of sudo +--- + performance-test/T03-count.sh | 23 +++++++++++++++++++++++ + 1 file changed, 23 insertions(+) + create mode 100755 performance-test/T03-count.sh + +diff --git a/performance-test/T03-count.sh b/performance-test/T03-count.sh +new file mode 100755 +index 0000000..e74437e +--- /dev/null ++++ b/performance-test/T03-count.sh +@@ -0,0 +1,23 @@ ++#!/bin/bash ++ ++test_description='counting' ++ ++. ./perf-test-lib.sh ++ ++time_start ++ ++for i in $(seq 1 10); do ++ echo 3 | sudo tee /proc/sys/vm/drop_caches ++ notmuch --version > /dev/null ++ time_run "from:torvalds@linux-foundation.org " \ ++ "notmuch count from:torvalds@linux-foundation.org" ++ ++ echo 3 | sudo tee /proc/sys/vm/drop_caches ++ notmuch --version > /dev/null ++ ++ time_run "from:linux-foundation.org " \ ++ "notmuch count from:linux-foundation.org" ++done ++ ++ ++time_done +-- +1.9.1 + + +--=-=-=-- -- 2.26.2