Re: slowdown in notmuch perf suite with xapian 1.3.5

diff --git a/77/8f943c4e71e051c5aa6d5fe902fdfc64b0e487 b/77/8f943c4e71e051c5aa6d5fe902fdfc64b0e487
new file mode 100644 (file)
index 0000000..d325595
--- /dev/null
+++ b/77/8f943c4e71e051c5aa6d5fe902fdfc64b0e487
@@ -0,0 +1,98 @@
+Return-Path: <david@tethera.net>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+ by arlo.cworth.org (Postfix) with ESMTP id 590F56DE02B5\r
+ for <notmuch@notmuchmail.org>; Thu,  7 Apr 2016 18:14:36 -0700 (PDT)\r
+X-Virus-Scanned: Debian amavisd-new at cworth.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -0.02\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-0.02 tagged_above=-999 required=5 tests=[AWL=-0.009,\r
+  SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=disabled\r
+Received: from arlo.cworth.org ([127.0.0.1])\r
+ by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024)\r
+ with ESMTP id yW41mXuYw6UY for <notmuch@notmuchmail.org>;\r
+ Thu,  7 Apr 2016 18:14:27 -0700 (PDT)\r
+Received: from fethera.tethera.net (fethera.tethera.net [198.245.60.197])\r
+ by arlo.cworth.org (Postfix) with ESMTPS id 8E5E86DE0134\r
+ for <notmuch@notmuchmail.org>; Thu,  7 Apr 2016 18:14:26 -0700 (PDT)\r
+Received: from remotemail by fethera.tethera.net with local (Exim 4.84)\r
+ (envelope-from <david@tethera.net>)\r
+ id 1aoL0N-0000dI-M9; Thu, 07 Apr 2016 21:14:35 -0400\r
+Received: (nullmailer pid 2722 invoked by uid 1000);\r
+ Fri, 08 Apr 2016 01:14:24 -0000\r
+From: David Bremner <david@tethera.net>\r
+To: Olly Betts <olly@survex.com>\r
+Cc: notmuch@notmuchmail.org, xapian-discuss@lists.xapian.org\r
+Subject: Re: slowdown in notmuch perf suite with xapian 1.3.5\r
+In-Reply-To: <20160408005725.GA3037@survex.com>\r
+References: <87twjd639d.fsf@zancas.localnet>\r
+ <20160407232537.GB29434@survex.com> <87h9fd53vo.fsf@zancas.localnet>\r
+ <20160408005725.GA3037@survex.com>\r
+User-Agent: Notmuch/0.21+99~gd93d377 (http://notmuchmail.org) Emacs/24.5.1\r
+ (x86_64-pc-linux-gnu)\r
+Date: Thu, 07 Apr 2016 22:14:24 -0300\r
+Message-ID: <87egag6gwf.fsf@zancas.localnet>\r
+MIME-Version: 1.0\r
+Content-Type: text/plain\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.20\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+ <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <https://notmuchmail.org/mailman/options/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch/>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <https://notmuchmail.org/mailman/listinfo/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Fri, 08 Apr 2016 01:14:36 -0000\r
+\r
+Olly Betts <olly@survex.com> writes:\r
+\r
+> Non-cached reads/writes are arguably the most useful sort to measure, but the\r
+> reads at least will be sensitive to OS caching, which means a repeat run will\r
+> generally show lower numbers of reads, e.g.:\r
+>\r
+> $ /usr/bin/time -f '%I/%O' wc randomfile \r
+>   240  2908 96780 randomfile\r
+> 192/0\r
+> $ /usr/bin/time -f '%I/%O' wc randomfile \r
+>   240  2908 96780 randomfile\r
+> 0/0\r
+>\r
+> So those numbers may not be entirely comparable, depending what order your\r
+> tests were done in, and whether you'd run the tests (or cloned the repo or some\r
+> other operation which read or wrote the files used) recently enough that their\r
+> data might still be cached.\r
+\r
+Here are the number from second glass run. The order was glass / chert /\r
+glass\r
+\r
+\r
+T00-new.sh: Testing notmuch new                         [0.4 large]\r
+                       Wall(s) Usr(s)  Sys(s)  Res(K)  In/Out(512B)\r
+  Initial notmuch new   920.53 698.96  207.02  245188  3528/22442096\r
+  notmuch new #2        0.55   0.00    0.01    8048    6960/160\r
+  notmuch new #3        0.01   0.00    0.00    8112    0/8\r
+  notmuch new #4        0.01   0.01    0.00    8136    0/8\r
+  notmuch new #5        0.01   0.00    0.00    8140    0/8\r
+  notmuch new #6        0.01   0.00    0.00    8116    0/8\r
+\r
+T01-dump-restore.sh: Testing dump and restore           [0.4 large]\r
+                       Wall(s) Usr(s)  Sys(s)  Res(K)  In/Out(512B)\r
+  load nmbug tags       8.89   4.23    3.88    11648   368/40072\r
+  dump *                7.37   6.29    1.08    25268   72/27928\r
+  restore *             7.60   7.16    0.43    8624    0/0\r
+\r
+T02-tag.sh: Testing tagging                             [0.4 large]\r
+                       Wall(s) Usr(s)  Sys(s)  Res(K)  In/Out(512B)\r
+  tag * +new_tag        474.16 274.89  191.52  34820   16/1920240\r
+  tag * +existing_tag   0.01   0.01    0.00    8480    152/0\r
+  tag * -existing_tag   438.62 239.02  195.44  34928   0/1970160\r
+  tag * -missing_tag    0.00   0.00    0.00    8264    0/0\r
+\r
+It's a bit faster overall, but not radically so. So I think cache\r
+effects are not the main issue here.\r