Re: [PATCH 0/3] Outline fix for emacs tagging race

diff --git a/30/6931dbcbee3b3d1a557b3c40144e8c4fbbdfce b/30/6931dbcbee3b3d1a557b3c40144e8c4fbbdfce
new file mode 100644 (file)
index 0000000..d113e8e
--- /dev/null
+++ b/30/6931dbcbee3b3d1a557b3c40144e8c4fbbdfce
@@ -0,0 +1,121 @@
+Return-Path: <tomi.ollila@iki.fi>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+       by olra.theworths.org (Postfix) with ESMTP id 840DC431FC3\r
+       for <notmuch@notmuchmail.org>; Fri,  9 Nov 2012 13:26:02 -0800 (PST)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: 0\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=0 tagged_above=-999 required=5 tests=[none]\r
+       autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+       by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+       with ESMTP id yveKxeW6y-Bj for <notmuch@notmuchmail.org>;\r
+       Fri,  9 Nov 2012 13:26:00 -0800 (PST)\r
+Received: from guru.guru-group.fi (guru.guru-group.fi [46.183.73.34])\r
+       by olra.theworths.org (Postfix) with ESMTP id 0ECE7431FAE\r
+       for <notmuch@notmuchmail.org>; Fri,  9 Nov 2012 13:26:00 -0800 (PST)\r
+Received: from guru.guru-group.fi (localhost [IPv6:::1])\r
+       by guru.guru-group.fi (Postfix) with ESMTP id CA202100094;\r
+       Fri,  9 Nov 2012 23:25:58 +0200 (EET)\r
+From: Tomi Ollila <tomi.ollila@iki.fi>\r
+To: Mark Walters <markwalters1009@gmail.com>, notmuch@notmuchmail.org\r
+Subject: Re: [PATCH 0/3] Outline fix for emacs tagging race\r
+In-Reply-To: <1352487491-31512-1-git-send-email-markwalters1009@gmail.com>\r
+References: <1352487491-31512-1-git-send-email-markwalters1009@gmail.com>\r
+User-Agent: Notmuch/0.14+84~g8a199bf (http://notmuchmail.org) Emacs/24.2.1\r
+       (x86_64-unknown-linux-gnu)\r
+X-Face: HhBM'cA~<r"^Xv\KRN0P{vn'Y"Kd;zg_y3S[4)KSN~s?O\"QPoL\r
+       $[Xv_BD:i/F$WiEWax}R(MPS`^UaptOGD`*/=@\1lKoVa9tnrg0TW?"r7aRtgk[F\r
+       !)g;OY^,BjTbr)Np:%c_o'jj,Z\r
+Date: Fri, 09 Nov 2012 23:25:58 +0200\r
+Message-ID: <m2mwyqjwnt.fsf@guru.guru-group.fi>\r
+MIME-Version: 1.0\r
+Content-Type: text/plain\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+       <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Fri, 09 Nov 2012 21:26:03 -0000\r
+\r
+On Fri, Nov 09 2012, Mark Walters <markwalters1009@gmail.com> wrote:\r
+\r
+>\r
+> This patch series implements the above mentioned solution and seems to\r
+> work except for one problem.\r
+>\r
+> Since emacs now tags each message in a thread in the search buffer it\r
+> is easy to ask it to tag a lot of messages. This could be done\r
+> individually which would be ridiculously slow so instead they are all\r
+> done in one batch. But now it is relatively easy to take notmuch over\r
+> the threshold for ARG_MAX.\r
+>\r
+> In [3] Tomi did some experiments and found on a standard Debian system\r
+> with getconf ARG_MAX =131072 that command lines with 10000 200 byte\r
+> arguments worked. I am a little puzzled by that as I get the same\r
+> results and I getconf ARG_MAX gives 2097152 for me.\r
+>\r
+> More importantly though, when trying to execute a command from emacs I\r
+> am finding that 131072 is the limit on the command length in bytes and\r
+> we can hit this with something around 1500 messages (see end for a\r
+> very hacky emacs test script). This is probably more than we can\r
+> expect in a single thread so tagging from show is probably safe but it\r
+> does cause a problem when tagging from search.\r
+\r
+That's because your test script below constructs one very loooong\r
+string (and used just one arg). Like you said, [3] used 10000 200-char args.\r
+\r
+This test script is more like what the perl(1) script did in [3]\r
+\r
+(progn\r
+  (setq arglist '("aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"))\r
+  (setq n 10000)\r
+  (while (> n 0)\r
+    (setq n (1- n))\r
+    (setq arglist (cons "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb" arglist)))\r
+\r
+  (apply 'call-process "/bin/echo" nil t nil arglist))\r
+\r
+(except that the args here are just 80 chars but you should get the point :)\r
+\r
+\r
+I did not check your patches yet.. buf if you concatenate there you \r
+could try to provide those as separate args...\r
+\r
+... then we can experiment how notmuch (and xapian) copes with thosew\r
+IIRC notmuch will concatenate one long string out of those and do\r
+xapian stuff using that (but then we're not limited by command line\r
+arguments restrictions).\r
+\r
+>\r
+> Best wishes \r
+>\r
+> Mark\r
+\r
+Tomi\r
+\r
+>\r
+> [3] id:m2liody7av.fsf@guru.guru-group.fi\r
+\r
+> TEST SCRIPT\r
+\r
+> (progn\r
+>   (setq bigstring\r
+> "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa")\r
+>   (setq n 1310)\r
+>   (setq big nil)\r
+>   (while (> n 0)\r
+>    (setq n (1- n))\r
+>     (setq big (concat big (format "%s" n) " " bigstring)))\r
+>   (call-process "echo" nil t nil big))\r
+\r