From d9f4cdbafb58adc1ed35d5e48b77bf2a7db2b6e5 Mon Sep 17 00:00:00 2001 From: Tomi Ollila Date: Wed, 8 Feb 2012 11:59:36 +0200 Subject: [PATCH] Re: [PATCH v3] emacs: Call "notmuch tag" once when applying tag changes to a thread. --- 7f/bbbacae1ad70252595f9e2b8fc1cc666207c35 | 130 ++++++++++++++++++++++ 1 file changed, 130 insertions(+) create mode 100644 7f/bbbacae1ad70252595f9e2b8fc1cc666207c35 diff --git a/7f/bbbacae1ad70252595f9e2b8fc1cc666207c35 b/7f/bbbacae1ad70252595f9e2b8fc1cc666207c35 new file mode 100644 index 000000000..63e219e79 --- /dev/null +++ b/7f/bbbacae1ad70252595f9e2b8fc1cc666207c35 @@ -0,0 +1,130 @@ +Return-Path: +X-Original-To: notmuch@notmuchmail.org +Delivered-To: notmuch@notmuchmail.org +Received: from localhost (localhost [127.0.0.1]) + by olra.theworths.org (Postfix) with ESMTP id 8EBEB431FAF + for ; Wed, 8 Feb 2012 01:59:38 -0800 (PST) +X-Virus-Scanned: Debian amavisd-new at olra.theworths.org +X-Spam-Flag: NO +X-Spam-Score: 0 +X-Spam-Level: +X-Spam-Status: No, score=0 tagged_above=-999 required=5 tests=[none] + autolearn=disabled +Received: from olra.theworths.org ([127.0.0.1]) + by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) + with ESMTP id kEOkO54WAUez for ; + Wed, 8 Feb 2012 01:59:37 -0800 (PST) +Received: from guru.guru-group.fi (guru-group.fi [87.108.86.66]) + by olra.theworths.org (Postfix) with ESMTP id B8D41431FAE + for ; Wed, 8 Feb 2012 01:59:36 -0800 (PST) +Received: by guru.guru-group.fi (Postfix, from userid 501) + id ADC3668055; Wed, 8 Feb 2012 11:59:36 +0200 (EET) +From: Tomi Ollila +To: David Edmondson , notmuch@notmuchmail.org +Subject: Re: [PATCH v3] emacs: Call "notmuch tag" once when applying tag + changes to a thread. +In-Reply-To: <1328690169-6991-1-git-send-email-dme@dme.org> +References: <1328632303-31877-1-git-send-email-jani@nikula.org> + <1328690169-6991-1-git-send-email-dme@dme.org> +User-Agent: Notmuch/0.11.1+164~g6619341 (http://notmuchmail.org) Emacs/23.3.1 + (x86_64-unknown-linux-gnu) +X-Face: HhBM'cA~ +MIME-Version: 1.0 +Content-Type: text/plain; charset=us-ascii +X-BeenThere: notmuch@notmuchmail.org +X-Mailman-Version: 2.1.13 +Precedence: list +List-Id: "Use and development of the notmuch mail system." + +List-Unsubscribe: , + +List-Archive: +List-Post: +List-Help: +List-Subscribe: , + +X-List-Received-Date: Wed, 08 Feb 2012 09:59:38 -0000 + +On Wed, 8 Feb 2012 08:36:09 +0000, David Edmondson wrote: +> Optimize thread tagging by combining all the tagging operations to a +> single "notmuch tag" call. +> +> For threads in the order of tens or a hundred inbox tagged messages, +> this gives a noticeable speedup. On two different machines, archiving +> a thread of about 50 inbox tagged messages goes down from 10+ seconds +> to about 0.5 seconds. +> +> The bottleneck is not within emacs; the same behaviour can be observed +> in the CLI. This approach has the added benefit of being more +> reliable: any of the individual tagging operations might face a locked +> database, leading to partial results. +> +> This introduces a limitation to the number of messages that can be +> archived at the same time (through ARG_MAX limiting the command +> line). While at least on Linux this seems more like a theoretical +> limitation than a real one, it could be avoided by archiving at most a +> few hundred messages at a time. + +I did a simple test program: + +--8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<-- +#!/usr/bin/perl + +use strict; +use warnings; + +die "Usage: $0 <# of args>\n" unless @ARGV == 2; + +my $arg = 'x' x $ARGV[0]; +my @args = ( $arg ) x $ARGV[1]; + +print "One arg: '$arg'\n"; +print 'Number of args: ', scalar @args, "\n"; + +exec '/bin/true', @args; +--8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<-- + +This program executes /bin/true with given number of args all args +of some length: i.e. + +./test_cmdlimit 100 10000 + +makes 10000 100 character arguments (total of million characters) +and executes /bin/true with those ten thousand 100-char args. + +In one machine where getconf ARG_MAX returns 131072 +(Debian Lenny ia32) + +./test_cmdlimit 200 10000 succeeds +but +./test_cmdlimit 20 100000 gives +Can't exec "/bin/true": Argument list too long at ./test_cmdlimit.pl line 14. + +Hmm, actually there: +./test_cmdlimit 19 100000 succeeds and +./test_cmdlimit.pl 209 10000 fails + +More with args which lengths are 1 and 2 chars: + +./test_cmdlimit.pl 1 1046163 succeeds +./test_cmdlimit.pl 1 1046164 fails + +./test_cmdlimit.pl 2 697442 succeeds +./test_cmdlimit.pl 2 697443 fails + +1046163 was close to 1048576 (1024 * 1024) -- +697442 * 2 is 1394884... + +>From these I can make an educated guess that when message id:s +are typically between 30 to 70 characters something like +20 000 messages are safe to be tagged at once in this +test system (./test_cmdlimit.pl 50 40000 succeeds). + +> +> Based on code from Jani Nikula . + +Tomi -- 2.26.2