From 5b5b22229d0fc31628aedff15298edf564ca0d5e Mon Sep 17 00:00:00 2001 From: Tomi Ollila Date: Fri, 9 Nov 2012 23:25:58 +0200 Subject: [PATCH] Re: [PATCH 0/3] Outline fix for emacs tagging race --- 30/6931dbcbee3b3d1a557b3c40144e8c4fbbdfce | 121 ++++++++++++++++++++++ 1 file changed, 121 insertions(+) create mode 100644 30/6931dbcbee3b3d1a557b3c40144e8c4fbbdfce diff --git a/30/6931dbcbee3b3d1a557b3c40144e8c4fbbdfce b/30/6931dbcbee3b3d1a557b3c40144e8c4fbbdfce new file mode 100644 index 000000000..d113e8e6c --- /dev/null +++ b/30/6931dbcbee3b3d1a557b3c40144e8c4fbbdfce @@ -0,0 +1,121 @@ +Return-Path: +X-Original-To: notmuch@notmuchmail.org +Delivered-To: notmuch@notmuchmail.org +Received: from localhost (localhost [127.0.0.1]) + by olra.theworths.org (Postfix) with ESMTP id 840DC431FC3 + for ; Fri, 9 Nov 2012 13:26:02 -0800 (PST) +X-Virus-Scanned: Debian amavisd-new at olra.theworths.org +X-Spam-Flag: NO +X-Spam-Score: 0 +X-Spam-Level: +X-Spam-Status: No, score=0 tagged_above=-999 required=5 tests=[none] + autolearn=disabled +Received: from olra.theworths.org ([127.0.0.1]) + by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) + with ESMTP id yveKxeW6y-Bj for ; + Fri, 9 Nov 2012 13:26:00 -0800 (PST) +Received: from guru.guru-group.fi (guru.guru-group.fi [46.183.73.34]) + by olra.theworths.org (Postfix) with ESMTP id 0ECE7431FAE + for ; Fri, 9 Nov 2012 13:26:00 -0800 (PST) +Received: from guru.guru-group.fi (localhost [IPv6:::1]) + by guru.guru-group.fi (Postfix) with ESMTP id CA202100094; + Fri, 9 Nov 2012 23:25:58 +0200 (EET) +From: Tomi Ollila +To: Mark Walters , notmuch@notmuchmail.org +Subject: Re: [PATCH 0/3] Outline fix for emacs tagging race +In-Reply-To: <1352487491-31512-1-git-send-email-markwalters1009@gmail.com> +References: <1352487491-31512-1-git-send-email-markwalters1009@gmail.com> +User-Agent: Notmuch/0.14+84~g8a199bf (http://notmuchmail.org) Emacs/24.2.1 + (x86_64-unknown-linux-gnu) +X-Face: HhBM'cA~ +MIME-Version: 1.0 +Content-Type: text/plain +X-BeenThere: notmuch@notmuchmail.org +X-Mailman-Version: 2.1.13 +Precedence: list +List-Id: "Use and development of the notmuch mail system." + +List-Unsubscribe: , + +List-Archive: +List-Post: +List-Help: +List-Subscribe: , + +X-List-Received-Date: Fri, 09 Nov 2012 21:26:03 -0000 + +On Fri, Nov 09 2012, Mark Walters wrote: + +> +> This patch series implements the above mentioned solution and seems to +> work except for one problem. +> +> Since emacs now tags each message in a thread in the search buffer it +> is easy to ask it to tag a lot of messages. This could be done +> individually which would be ridiculously slow so instead they are all +> done in one batch. But now it is relatively easy to take notmuch over +> the threshold for ARG_MAX. +> +> In [3] Tomi did some experiments and found on a standard Debian system +> with getconf ARG_MAX =131072 that command lines with 10000 200 byte +> arguments worked. I am a little puzzled by that as I get the same +> results and I getconf ARG_MAX gives 2097152 for me. +> +> More importantly though, when trying to execute a command from emacs I +> am finding that 131072 is the limit on the command length in bytes and +> we can hit this with something around 1500 messages (see end for a +> very hacky emacs test script). This is probably more than we can +> expect in a single thread so tagging from show is probably safe but it +> does cause a problem when tagging from search. + +That's because your test script below constructs one very loooong +string (and used just one arg). Like you said, [3] used 10000 200-char args. + +This test script is more like what the perl(1) script did in [3] + +(progn + (setq arglist '("aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa")) + (setq n 10000) + (while (> n 0) + (setq n (1- n)) + (setq arglist (cons "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb" arglist))) + + (apply 'call-process "/bin/echo" nil t nil arglist)) + +(except that the args here are just 80 chars but you should get the point :) + + +I did not check your patches yet.. buf if you concatenate there you +could try to provide those as separate args... + +... then we can experiment how notmuch (and xapian) copes with thosew +IIRC notmuch will concatenate one long string out of those and do +xapian stuff using that (but then we're not limited by command line +arguments restrictions). + +> +> Best wishes +> +> Mark + +Tomi + +> +> [3] id:m2liody7av.fsf@guru.guru-group.fi + +> TEST SCRIPT + +> (progn +> (setq bigstring +> "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa") +> (setq n 1310) +> (setq big nil) +> (while (> n 0) +> (setq n (1- n)) +> (setq big (concat big (format "%s" n) " " bigstring))) +> (call-process "echo" nil t nil big)) + -- 2.26.2