From a75a8db87557bb4f8fad06b2d6b1f02e341d88b5 Mon Sep 17 00:00:00 2001 From: Austin Clements Date: Thu, 10 Oct 2013 10:14:24 +2000 Subject: [PATCH] Re: [PATCH 07/11] emacs: Use notmuch tag --batch for large tag queries --- c2/4b20d03341e7f10f0de6f6a90a847c1f015c72 | 157 ++++++++++++++++++++++ 1 file changed, 157 insertions(+) create mode 100644 c2/4b20d03341e7f10f0de6f6a90a847c1f015c72 diff --git a/c2/4b20d03341e7f10f0de6f6a90a847c1f015c72 b/c2/4b20d03341e7f10f0de6f6a90a847c1f015c72 new file mode 100644 index 000000000..b35648e7e --- /dev/null +++ b/c2/4b20d03341e7f10f0de6f6a90a847c1f015c72 @@ -0,0 +1,157 @@ +Return-Path: +X-Original-To: notmuch@notmuchmail.org +Delivered-To: notmuch@notmuchmail.org +Received: from localhost (localhost [127.0.0.1]) + by olra.theworths.org (Postfix) with ESMTP id 464EA431FAF + for ; Wed, 9 Oct 2013 07:14:33 -0700 (PDT) +X-Virus-Scanned: Debian amavisd-new at olra.theworths.org +X-Spam-Flag: NO +X-Spam-Score: -0.7 +X-Spam-Level: +X-Spam-Status: No, score=-0.7 tagged_above=-999 required=5 + tests=[RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled +Received: from olra.theworths.org ([127.0.0.1]) + by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) + with ESMTP id QI2Cz-hAIl6c for ; + Wed, 9 Oct 2013 07:14:28 -0700 (PDT) +Received: from dmz-mailsec-scanner-6.mit.edu (dmz-mailsec-scanner-6.mit.edu + [18.7.68.35]) + by olra.theworths.org (Postfix) with ESMTP id 69794431FAE + for ; Wed, 9 Oct 2013 07:14:28 -0700 (PDT) +X-AuditID: 12074423-b7fc98e0000009a2-5c-525564c306d9 +Received: from mailhub-auth-3.mit.edu ( [18.9.21.43]) + by dmz-mailsec-scanner-6.mit.edu (Symantec Messaging Gateway) with SMTP + id E7.CF.02466.3C465525; Wed, 9 Oct 2013 10:14:27 -0400 (EDT) +Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11]) + by mailhub-auth-3.mit.edu (8.13.8/8.9.2) with ESMTP id r99EEQHb006239; + Wed, 9 Oct 2013 10:14:27 -0400 +Received: from awakening.csail.mit.edu (awakening.csail.mit.edu [18.26.4.91]) + (authenticated bits=0) + (User authenticated as amdragon@ATHENA.MIT.EDU) + by outgoing.mit.edu (8.13.8/8.12.4) with ESMTP id r99EEPCp021588 + (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); + Wed, 9 Oct 2013 10:14:26 -0400 +Received: from amthrax by awakening.csail.mit.edu with local (Exim 4.80) + (envelope-from ) + id 1VTuWu-00050f-UC; Wed, 09 Oct 2013 10:14:24 -0400 +Date: Wed, 9 Oct 2013 10:14:24 -0400 +From: Austin Clements +To: Mark Walters +Subject: Re: [PATCH 07/11] emacs: Use notmuch tag --batch for large tag + queries +Message-ID: <20131009141424.GP21611@mit.edu> +References: <1381185201-25197-1-git-send-email-amdragon@mit.edu> + <1381185201-25197-8-git-send-email-amdragon@mit.edu> + <87iox6ap58.fsf@nikula.org> <87d2ne28sj.fsf@qmul.ac.uk> +MIME-Version: 1.0 +Content-Type: text/plain; charset=us-ascii +Content-Disposition: inline +In-Reply-To: <87d2ne28sj.fsf@qmul.ac.uk> +User-Agent: Mutt/1.5.21 (2010-09-15) +X-Brightmail-Tracker: + H4sIAAAAAAAAA+NgFuphleLIzCtJLcpLzFFi42IR4hTV1j2cEhpksPSglkXTdGeL1XN5LK7f + nMnswOyxc9Zddo9b91+zezxbdYs5gDmKyyYlNSezLLVI3y6BK6P55VKmgq+SFTum9rM2MG4Q + 6WLk5JAQMJHoP3eLDcIWk7hwbz2QzcUhJLCPUeLn/A52CGcDo8SP+2+ZIZxTTBLHtzSyQjhL + GCXmr+1hBOlnEVCReNc1CWwWm4CGxLb9y8HiIgI6ErcPLWAHsZkFrCQatnwAiwsLBEpc3dwO + NIiDgxeoZk5jGcTMzYwSe45OYgap4RUQlDg58wkLRK+WxI1/L5lA6pkFpCWW/+MACXMCrbrS + 28MKYosCnTDl5Da2CYxCs5B0z0LSPQuhewEj8ypG2ZTcKt3cxMyc4tRk3eLkxLy81CJdM73c + zBK91JTSTYygQGd3Ud7B+Oeg0iFGAQ5GJR7eDt6QICHWxLLiytxDjJIcTEqivNtjQoOE+JLy + UyozEosz4otKc1KLDzFKcDArifAaRwPleFMSK6tSi/JhUtIcLErivLc47IOEBNITS1KzU1ML + UotgsjIcHEoSvD3JQI2CRanpqRVpmTklCGkmDk6Q4TxAw7eB1PAWFyTmFmemQ+RPMSpKifMe + B0kIgCQySvPgemGJ6BWjONArwry7QKp4gEkMrvsV0GAmoMHbv4eADC5JREhJNTAGrP+oJt/w + WOPYf5EIE73sPTcPfyyq3pW2dddB5QXWez8miL4+LrQhdFtc9fSieo3ffbu9D0keuD9fTvpG + 6dfe3BlXZbZtecki/mfLVSbO1D8yn9m7v9wPf8J80TxaymPdQ53lmleWdU15KMD8/2J3+7Nj + q08/Clc4udDv4fpujsi9z5R7O7jMlViKMxINtZiLihMB5Mwz8x8DAAA= +Cc: notmuch@notmuchmail.org +X-BeenThere: notmuch@notmuchmail.org +X-Mailman-Version: 2.1.13 +Precedence: list +List-Id: "Use and development of the notmuch mail system." + +List-Unsubscribe: , + +List-Archive: +List-Post: +List-Help: +List-Subscribe: , + +X-List-Received-Date: Wed, 09 Oct 2013 14:14:33 -0000 + +Quoth Mark Walters on Oct 09 at 8:38 am: +> +> On Wed, 09 Oct 2013, Jani Nikula wrote: +> > On Tue, 08 Oct 2013, Austin Clements wrote: +> >> (Unfortunately, it's difficult to first demonstrate this problem with +> >> a known-broken test because modern Linux kernels have argument length +> >> limits in the megabytes, which makes Emacs really slow!) +> >> --- +> >> emacs/notmuch-lib.el | 8 ++++++++ +> >> emacs/notmuch-tag.el | 12 ++++++++++-- +> >> test/emacs | 8 ++++++++ +> >> 3 files changed, 26 insertions(+), 2 deletions(-) +> >> +> >> diff --git a/emacs/notmuch-lib.el b/emacs/notmuch-lib.el +> >> index 22156f1..348112b 100644 +> >> --- a/emacs/notmuch-lib.el +> >> +++ b/emacs/notmuch-lib.el +> >> @@ -261,6 +261,14 @@ user-friendly queries." +> >> "Return a query that matches the message with id ID." +> >> (concat "id:" (notmuch-escape-boolean-term id))) +> >> +> >> +(defun notmuch-hex-encode (str) +> >> + "Hex-encode STR (e.g., as used by batch tagging). +> >> + +> >> +This replaces spaces, percents, and double quotes in STR with +> >> +%NN where NN is the hexadecimal value of the character." +> >> + (replace-regexp-in-string +> >> + "[ %\"]" (lambda (match) (format "%%%02x" (aref match 0))) str)) +> >> + +> >> ;; +> >> +> >> (defun notmuch-common-do-stash (text) +> >> diff --git a/emacs/notmuch-tag.el b/emacs/notmuch-tag.el +> >> index 064cfa8..a4eec14 100644 +> >> --- a/emacs/notmuch-tag.el +> >> +++ b/emacs/notmuch-tag.el +> >> @@ -242,6 +242,8 @@ from TAGS if present." +> >> (error "Changed tag must be of the form `+this_tag' or `-that_tag'"))))) +> >> (sort result-tags 'string<))) +> >> +> >> +(defconst notmuch-tag-argument-limit 1000) +> >> + +> >> (defun notmuch-tag (query &optional tag-changes) +> >> "Add/remove tags in TAG-CHANGES to messages matching QUERY. +> >> +> >> @@ -268,8 +270,14 @@ notmuch-after-tag-hook will be run." +> >> tag-changes) +> >> (unless (null tag-changes) +> >> (run-hooks 'notmuch-before-tag-hook) +> >> - (apply 'notmuch-call-notmuch-process "tag" +> >> - (append tag-changes (list "--" query))) +> >> + (if (<= (length query) notmuch-tag-argument-limit) +> >> + (apply 'notmuch-call-notmuch-process "tag" +> >> + (append tag-changes (list "--" query))) +> >> + ;; Use batch tag mode to avoid argument length limitations +> >> + (let ((batch-op (concat (mapconcat #'notmuch-hex-encode tag-changes " ") +> >> + " -- " query))) +> >> + (message "Batch tagging with %s" batch-op) +> > +> > Why the message? +> > +> > Jani. +> +> I had missed that: this message is presumably for debugging as it +> includes the (possibly megabyte sized) query. I think some message would +> be sensible as this could be a slow operation so it's probably worth +> telling the user why the interface has locked. + +You're right, this was a leftover debug message. While I wouldn't +mind a progress message for long tagging operations, query length +isn't a good approximation of how long it's going to take. I was +planning to include messages on tagging operations in my undo series, +which I think can do it in a more principled way (assuming I can find +a clean way to update tags in the UI on undo). + +> Best wishes +> +> Mark -- 2.26.2