--- /dev/null
+Return-Path: <markwalters1009@gmail.com>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+ by olra.theworths.org (Postfix) with ESMTP id 678C0431FB6\r
+ for <notmuch@notmuchmail.org>; Fri, 9 Nov 2012 10:58:27 -0800 (PST)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: 0.201\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=0.201 tagged_above=-999 required=5\r
+ tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1,\r
+ FREEMAIL_ENVFROM_END_DIGIT=1, FREEMAIL_FROM=0.001,\r
+ RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+ by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+ with ESMTP id pp1zC4URjimz for <notmuch@notmuchmail.org>;\r
+ Fri, 9 Nov 2012 10:58:25 -0800 (PST)\r
+Received: from mail-wg0-f45.google.com (mail-wg0-f45.google.com\r
+ [74.125.82.45]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client\r
+ certificate requested) by olra.theworths.org (Postfix) with ESMTPS id\r
+ 34FC2431FAE for <notmuch@notmuchmail.org>; Fri, 9 Nov 2012 10:58:25 -0800\r
+ (PST)\r
+Received: by mail-wg0-f45.google.com with SMTP id dq12so2288423wgb.2\r
+ for <notmuch@notmuchmail.org>; Fri, 09 Nov 2012 10:58:23 -0800 (PST)\r
+DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;\r
+ h=from:to:cc:subject:date:message-id:x-mailer;\r
+ bh=iVqVh7xUHX1VPSG0tXROjHgC34l3GXEyAxVmiyl7UI0=;\r
+ b=Ak3HOehNKJA4AqbhdaDgL2jJXzmlmY2BcR9rJT3FaFviPiWvvZuP682qIxK95qn6CG\r
+ UTX29l7j1NhYK3vEbn2/Pq/rISpq2y/uFas3qhllSf1AoIs/v1Nejsvh0JOZ+/CwW4gx\r
+ ZN49TadE0ISl8JOfs4n1jxbBdxom3TSEOpJKUwPmOIlikGBRG9ber0I6uh41CTuDXb+l\r
+ Nm8/sCg6vWjnffYSs3eAL5OCxKnenoXJsaS2RItEJI206B1XiPqtuKeArOcpMjlOcyDR\r
+ A7gesBy05x79a8fafBJlr+XiIpjBBfO3Lp9p7OpRT/CJmh0M5ZTPKKjgaxK6Dd1jqYRX\r
+ IUhQ==\r
+Received: by 10.180.95.201 with SMTP id dm9mr4184109wib.3.1352487502655;\r
+ Fri, 09 Nov 2012 10:58:22 -0800 (PST)\r
+Received: from localhost (93-97-24-31.zone5.bethere.co.uk. [93.97.24.31])\r
+ by mx.google.com with ESMTPS id r10sm3779351wiz.0.2012.11.09.10.58.20\r
+ (version=TLSv1/SSLv3 cipher=OTHER);\r
+ Fri, 09 Nov 2012 10:58:21 -0800 (PST)\r
+From: Mark Walters <markwalters1009@gmail.com>\r
+To: notmuch@notmuchmail.org\r
+Subject: [PATCH 0/3] Outline fix for emacs tagging race\r
+Date: Fri, 9 Nov 2012 18:58:08 +0000\r
+Message-Id: <1352487491-31512-1-git-send-email-markwalters1009@gmail.com>\r
+X-Mailer: git-send-email 1.7.9.1\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+ <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Fri, 09 Nov 2012 18:58:27 -0000\r
+\r
+For a long time [1] there have been two related races in tagging from\r
+the search buffer.\r
+\r
+The first is that when tagging (including archiving) a thread message\r
+which arrived after the buffer was created may also be tagged. This is\r
+because the tagging is done based on the thread-id not on the\r
+individual messages.\r
+\r
+The second is when using the '*' command to tag all messages. This is\r
+not quite the same as this command only tags messages matching the\r
+query not all messages in all threads that contain a message matching\r
+the query. Thus if more messages now match than when the buffer was\r
+created (eg some external tagging script has run) then this command\r
+can unexpectedly tag these messages too.\r
+\r
+One solution that was discussed in [2] was for the search output of\r
+notmuch to include the message-ids of both matching and non-matching\r
+messages. At that time that was difficult to implement as it was\r
+unclear how to escape the message ids when using the text\r
+format. Since emacs now uses JSON for search mode this problem is\r
+solved.\r
+\r
+This patch series implements the above mentioned solution and seems to\r
+work except for one problem.\r
+\r
+Since emacs now tags each message in a thread in the search buffer it\r
+is easy to ask it to tag a lot of messages. This could be done\r
+individually which would be ridiculously slow so instead they are all\r
+done in one batch. But now it is relatively easy to take notmuch over\r
+the threshold for ARG_MAX.\r
+\r
+In [3] Tomi did some experiments and found on a standard Debian system\r
+with getconf ARG_MAX =131072 that command lines with 10000 200 byte\r
+arguments worked. I am a little puzzled by that as I get the same\r
+results and I getconf ARG_MAX gives 2097152 for me.\r
+\r
+More importantly though, when trying to execute a command from emacs I\r
+am finding that 131072 is the limit on the command length in bytes and\r
+we can hit this with something around 1500 messages (see end for a\r
+very hacky emacs test script). This is probably more than we can\r
+expect in a single thread so tagging from show is probably safe but it\r
+does cause a problem when tagging from search.\r
+\r
+I can think of several possible solutions (e.g., batch it in my new\r
+stuff, put some batching in notmuch-tag, all notmuch tag to read a\r
+query from stdin). But before any larger more intrusive change do\r
+people like the general approach? Does anyone have a good way to get\r
+round the command line size problem?\r
+\r
+Best wishes \r
+\r
+Mark\r
+\r
+\r
+[1] id:87ocmtg9ni.fsf@yoom.home.cworth.org\r
+[2] id:CAH-f9WticM4EN8F1_ik_-mcBcBtrXwSpO+Drbtp7=UN7McECrg@mail.gmail.com\r
+[3] id:m2liody7av.fsf@guru.guru-group.fi\r
+\r
+Mark Walters (3):\r
+ test: test for race when tagging from emacs search\r
+ cli: all search mode to include msg-ids with JSON output\r
+ emacs: make emacs use message-ids for tagging\r
+\r
+ emacs/notmuch.el | 22 ++++++++++++++++++++--\r
+ notmuch-search.c | 40 ++++++++++++++++++++++++++++++++++++++--\r
+ test/emacs | 21 +++++++++++++++++++++\r
+ 3 files changed, 79 insertions(+), 4 deletions(-)\r
+\r
+\r
+TEST SCRIPT\r
+\r
+(progn\r
+ (setq bigstring "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa")\r
+ (setq n 1310)\r
+ (setq big nil)\r
+ (while (> n 0)\r
+ (setq n (1- n))\r
+ (setq big (concat big (format "%s" n) " " bigstring)))\r
+ (call-process "echo" nil t nil big))\r
+\r
+\r
+\r
+-- \r
+1.7.9.1\r
+\r
projects / notmuch-archives.git / commitdiff