Re: [PATCH v3] emacs: Call "notmuch tag" once when applying tag changes to a thread.

diff --git a/7f/bbbacae1ad70252595f9e2b8fc1cc666207c35 b/7f/bbbacae1ad70252595f9e2b8fc1cc666207c35
new file mode 100644 (file)
index 0000000..63e219e
--- /dev/null
+++ b/7f/bbbacae1ad70252595f9e2b8fc1cc666207c35
@@ -0,0 +1,130 @@
+Return-Path: <tomi.ollila@iki.fi>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+       by olra.theworths.org (Postfix) with ESMTP id 8EBEB431FAF\r
+       for <notmuch@notmuchmail.org>; Wed,  8 Feb 2012 01:59:38 -0800 (PST)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: 0\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=0 tagged_above=-999 required=5 tests=[none]\r
+       autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+       by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+       with ESMTP id kEOkO54WAUez for <notmuch@notmuchmail.org>;\r
+       Wed,  8 Feb 2012 01:59:37 -0800 (PST)\r
+Received: from guru.guru-group.fi (guru-group.fi [87.108.86.66])\r
+       by olra.theworths.org (Postfix) with ESMTP id B8D41431FAE\r
+       for <notmuch@notmuchmail.org>; Wed,  8 Feb 2012 01:59:36 -0800 (PST)\r
+Received: by guru.guru-group.fi (Postfix, from userid 501)\r
+       id ADC3668055; Wed,  8 Feb 2012 11:59:36 +0200 (EET)\r
+From: Tomi Ollila <tomi.ollila@iki.fi>\r
+To: David Edmondson <dme@dme.org>, notmuch@notmuchmail.org\r
+Subject: Re: [PATCH v3] emacs: Call "notmuch tag" once when applying tag\r
+       changes to a thread.\r
+In-Reply-To: <1328690169-6991-1-git-send-email-dme@dme.org>\r
+References: <1328632303-31877-1-git-send-email-jani@nikula.org>\r
+       <1328690169-6991-1-git-send-email-dme@dme.org>\r
+User-Agent: Notmuch/0.11.1+164~g6619341 (http://notmuchmail.org) Emacs/23.3.1\r
+       (x86_64-unknown-linux-gnu)\r
+X-Face: HhBM'cA~<r"^Xv\KRN0P{vn'Y"Kd;zg_y3S[4)KSN~s?O\"QPoL\r
+       $[Xv_BD:i/F$WiEWax}R(MPS`^UaptOGD`*/=@\1lKoVa9tnrg0TW?"r7aRtgk[F\r
+       !)g;OY^,BjTbr)Np:%c_o'jj,Z\r
+Date: Wed, 08 Feb 2012 11:59:36 +0200\r
+Message-ID: <m2liody7av.fsf@guru.guru-group.fi>\r
+MIME-Version: 1.0\r
+Content-Type: text/plain; charset=us-ascii\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+       <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Wed, 08 Feb 2012 09:59:38 -0000\r
+\r
+On Wed,  8 Feb 2012 08:36:09 +0000, David Edmondson <dme@dme.org> wrote:\r
+> Optimize thread tagging by combining all the tagging operations to a\r
+> single "notmuch tag" call.\r
+> \r
+> For threads in the order of tens or a hundred inbox tagged messages,\r
+> this gives a noticeable speedup. On two different machines, archiving\r
+> a thread of about 50 inbox tagged messages goes down from 10+ seconds\r
+> to about 0.5 seconds.\r
+> \r
+> The bottleneck is not within emacs; the same behaviour can be observed\r
+> in the CLI. This approach has the added benefit of being more\r
+> reliable: any of the individual tagging operations might face a locked\r
+> database, leading to partial results.\r
+> \r
+> This introduces a limitation to the number of messages that can be\r
+> archived at the same time (through ARG_MAX limiting the command\r
+> line). While at least on Linux this seems more like a theoretical\r
+> limitation than a real one, it could be avoided by archiving at most a\r
+> few hundred messages at a time.\r
+\r
+I did a simple test program:\r
+\r
+--8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<--\r
+#!/usr/bin/perl\r
+\r
+use strict;\r
+use warnings;\r
+\r
+die "Usage: $0 <arglen> <# of args>\n" unless @ARGV == 2;\r
+\r
+my $arg = 'x' x $ARGV[0];\r
+my @args = ( $arg ) x $ARGV[1];\r
+\r
+print "One arg: '$arg'\n";\r
+print 'Number of args: ', scalar @args, "\n";\r
+\r
+exec '/bin/true', @args;\r
+--8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<--\r
+\r
+This program executes /bin/true with given number of args all args\r
+of some length: i.e.\r
+\r
+./test_cmdlimit 100 10000 \r
+\r
+makes 10000 100 character arguments (total of million characters)\r
+and executes /bin/true with those ten thousand 100-char args.\r
+\r
+In one machine where  getconf ARG_MAX  returns 131072\r
+(Debian Lenny ia32)\r
+\r
+./test_cmdlimit 200 10000   succeeds\r
+but\r
+./test_cmdlimit 20 100000   gives\r
+Can't exec "/bin/true": Argument list too long at ./test_cmdlimit.pl line 14.\r
+\r
+Hmm, actually there:\r
+./test_cmdlimit 19 100000   succeeds and\r
+./test_cmdlimit.pl 209 10000 fails\r
+\r
+More with args which lengths are 1 and 2 chars:\r
+\r
+./test_cmdlimit.pl 1 1046163 succeeds\r
+./test_cmdlimit.pl 1 1046164 fails\r
+\r
+./test_cmdlimit.pl 2 697442 succeeds\r
+./test_cmdlimit.pl 2 697443 fails\r
+\r
+1046163 was close to 1048576 (1024 * 1024) --\r
+697442 * 2 is 1394884...\r
+\r
+>From these I can make an educated guess that when message id:s\r
+are typically between 30 to 70 characters something like\r
+20 000 messages are safe to be tagged at once in this\r
+test system (./test_cmdlimit.pl 50 40000 succeeds).\r
+\r
+> \r
+> Based on code from Jani Nikula <jani@nikula.org>.\r
+\r
+Tomi\r