From: Olly Betts Date: Fri, 8 Apr 2016 00:57:25 +0000 (+0100) Subject: Re: slowdown in notmuch perf suite with xapian 1.3.5 X-Git-Url: http://git.tremily.us/?a=commitdiff_plain;h=0ea49b2c7378e5c53d2fa98507d10932f0aaf1b7;p=notmuch-archives.git Re: slowdown in notmuch perf suite with xapian 1.3.5 --- diff --git a/7f/65fbd68ff0dd697144fb8651add28d9572b22f b/7f/65fbd68ff0dd697144fb8651add28d9572b22f new file mode 100644 index 000000000..6efc8cc54 --- /dev/null +++ b/7f/65fbd68ff0dd697144fb8651add28d9572b22f @@ -0,0 +1,101 @@ +Return-Path: +X-Original-To: notmuch@notmuchmail.org +Delivered-To: notmuch@notmuchmail.org +Received: from localhost (localhost [127.0.0.1]) + by arlo.cworth.org (Postfix) with ESMTP id 5D4F26DE02B5 + for ; Thu, 7 Apr 2016 17:57:35 -0700 (PDT) +X-Virus-Scanned: Debian amavisd-new at cworth.org +X-Spam-Flag: NO +X-Spam-Score: -2.377 +X-Spam-Level: +X-Spam-Status: No, score=-2.377 tagged_above=-999 required=5 + tests=[AWL=-0.076, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001] + autolearn=disabled +Received: from arlo.cworth.org ([127.0.0.1]) + by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024) + with ESMTP id 7JG4OhwXFuzJ for ; + Thu, 7 Apr 2016 17:57:27 -0700 (PDT) +Received: from atreus.tartarus.org (atreus.tartarus.org [80.252.125.10]) + by arlo.cworth.org (Postfix) with ESMTPS id 8387A6DE0134 + for ; Thu, 7 Apr 2016 17:57:27 -0700 (PDT) +Received: from olly by atreus.tartarus.org with local (Exim 4.69) + (envelope-from ) + id 1aoKjl-0001nA-D6; Fri, 08 Apr 2016 01:57:25 +0100 +Date: Fri, 8 Apr 2016 01:57:25 +0100 +From: Olly Betts +To: David Bremner +Cc: notmuch@notmuchmail.org, xapian-discuss@lists.xapian.org +Subject: Re: slowdown in notmuch perf suite with xapian 1.3.5 +Message-ID: <20160408005725.GA3037@survex.com> +Reply-To: Xapian Discussion +Mail-Followup-To: David Bremner , + notmuch@notmuchmail.org, xapian-discuss@lists.xapian.org +References: <87twjd639d.fsf@zancas.localnet> + <20160407232537.GB29434@survex.com> + <87h9fd53vo.fsf@zancas.localnet> +MIME-Version: 1.0 +Content-Type: text/plain; charset=us-ascii +Content-Disposition: inline +In-Reply-To: <87h9fd53vo.fsf@zancas.localnet> +User-Agent: Mutt/1.5.21 (2010-09-15) +X-BeenThere: notmuch@notmuchmail.org +X-Mailman-Version: 2.1.20 +Precedence: list +List-Id: "Use and development of the notmuch mail system." + +List-Unsubscribe: , + +List-Archive: +List-Post: +List-Help: +List-Subscribe: , + +X-List-Received-Date: Fri, 08 Apr 2016 00:57:35 -0000 + +On Thu, Apr 07, 2016 at 09:40:59PM -0300, David Bremner wrote: +> Olly Betts writes: +> +> > +> > So the T00-new.sh numbers make sense - there's more work to do, and +> > we need to read existing positional data more to insert the new stuff, +> > so the increased reads and writes make sense. +> > +> > But guessing at what the other two tests do, I wouldn't expect them to +> > be affected by this. +> +> The non-optimized-away cases of T02-tag just adding and deleting terms +> to each document with term Tmail + +That should short-cut to just only changing the data for Tmail. Perhaps that's +not working correctly - I'll take a look at this, but probably after 1.4.0 is +out. + +> > I'm also a bit puzzled by how glass can manage not to read any data +> > for "dump *", and several tests seem to not read or write anything +> > for either backend. What exactly are the "In/Out" numbers? +> +> that's just the output from /usr/bin/time -f '%e\t%U\t%S\t%M\t%I/%O' +> +> The manual describes them as "number of file system +> inputs/outputs". From looking at the source, they correspond to +> ru_inblock and ru_oublock fields from the getrusage call. AFAIU, that +> means the number of non-cached read/writes. + +Non-cached reads/writes are arguably the most useful sort to measure, but the +reads at least will be sensitive to OS caching, which means a repeat run will +generally show lower numbers of reads, e.g.: + +$ /usr/bin/time -f '%I/%O' wc randomfile + 240 2908 96780 randomfile +192/0 +$ /usr/bin/time -f '%I/%O' wc randomfile + 240 2908 96780 randomfile +0/0 + +So those numbers may not be entirely comparable, depending what order your +tests were done in, and whether you'd run the tests (or cloned the repo or some +other operation which read or wrote the files used) recently enough that their +data might still be cached. + +Cheers, + Olly