--- /dev/null
+Return-Path: <amdragon@mit.edu>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+ by olra.theworths.org (Postfix) with ESMTP id 4BCFE431FB6\r
+ for <notmuch@notmuchmail.org>; Thu, 5 Jul 2012 11:58:31 -0700 (PDT)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -0.7\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-0.7 tagged_above=-999 required=5\r
+ tests=[RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+ by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+ with ESMTP id ys5Fpnb+gCD1 for <notmuch@notmuchmail.org>;\r
+ Thu, 5 Jul 2012 11:58:30 -0700 (PDT)\r
+Received: from dmz-mailsec-scanner-3.mit.edu (DMZ-MAILSEC-SCANNER-3.MIT.EDU\r
+ [18.9.25.14])\r
+ by olra.theworths.org (Postfix) with ESMTP id 24219431FAE\r
+ for <notmuch@notmuchmail.org>; Thu, 5 Jul 2012 11:58:30 -0700 (PDT)\r
+X-AuditID: 1209190e-b7fb56d0000008b2-cc-4ff5e3d48e1f\r
+Received: from mailhub-auth-4.mit.edu ( [18.7.62.39])\r
+ by dmz-mailsec-scanner-3.mit.edu (Symantec Messaging Gateway) with SMTP\r
+ id 7A.15.02226.4D3E5FF4; Thu, 5 Jul 2012 14:58:28 -0400 (EDT)\r
+Received: from outgoing.mit.edu (OUTGOING-AUTH.MIT.EDU [18.7.22.103])\r
+ by mailhub-auth-4.mit.edu (8.13.8/8.9.2) with ESMTP id q65IwRSL003765; \r
+ Thu, 5 Jul 2012 14:58:27 -0400\r
+Received: from awakening.csail.mit.edu (awakening.csail.mit.edu [18.26.4.91])\r
+ (authenticated bits=0)\r
+ (User authenticated as amdragon@ATHENA.MIT.EDU)\r
+ by outgoing.mit.edu (8.13.6/8.12.4) with ESMTP id q65IwQVx027141\r
+ (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NOT);\r
+ Thu, 5 Jul 2012 14:58:27 -0400 (EDT)\r
+Received: from amthrax by awakening.csail.mit.edu with local (Exim 4.77)\r
+ (envelope-from <amdragon@mit.edu>)\r
+ id 1SmrFy-00048a-C8; Thu, 05 Jul 2012 14:58:26 -0400\r
+Date: Thu, 5 Jul 2012 14:58:26 -0400\r
+From: Austin Clements <amdragon@MIT.EDU>\r
+To: Mark Walters <markwalters1009@gmail.com>\r
+Subject: Re: [PATCH 8/8] emacs: Switch from text to JSON format for search\r
+ results\r
+Message-ID: <20120705185826.GJ21653@mit.edu>\r
+References: <1341354059-29396-1-git-send-email-amdragon@mit.edu>\r
+ <1341354059-29396-9-git-send-email-amdragon@mit.edu>\r
+ <87y5myeh8g.fsf@qmul.ac.uk>\r
+MIME-Version: 1.0\r
+Content-Type: text/plain; charset=us-ascii\r
+Content-Disposition: inline\r
+In-Reply-To: <87y5myeh8g.fsf@qmul.ac.uk>\r
+User-Agent: Mutt/1.5.21 (2010-09-15)\r
+X-Brightmail-Tracker:\r
+ H4sIAAAAAAAAA+NgFupjleLIzCtJLcpLzFFi42IRYrdT173y+Ku/wY81ahar5/JYXL85k9ni\r
+ zcp5rA7MHjtn3WX3OPx1IYvHs1W3mAOYo7hsUlJzMstSi/TtErgy+mZsYi04bVIxr2EKWwPj\r
+ U/UuRk4OCQETie8T7rJC2GISF+6tZ+ti5OIQEtjHKHGtfTM7hLOeUeJJ/0wo5wSTxMNZW1kh\r
+ nCWMEr+e9DGD9LMIqEhMv/QCzGYT0JDYtn85I4gtIqAjcfvQAnYQm1lAX2LlyZlgNcICIRIL\r
+ z39nArF5gWpuHVrIDDF0KqPEuSdvGCESghInZz5hgWjWkrjx7yVQAweQLS2x/B8HSJgTaNfX\r
+ 9kawclGgG6ac3MY2gVFoFpLuWUi6ZyF0L2BkXsUom5JbpZubmJlTnJqsW5ycmJeXWqRrrJeb\r
+ WaKXmlK6iREU7JySfDsYvx5UOsQowMGoxMNrnPvFX4g1say4MvcQoyQHk5Io7+urX/2F+JLy\r
+ UyozEosz4otKc1KLDzFKcDArifD2ZgDleFMSK6tSi/JhUtIcLErivFdSbvoLCaQnlqRmp6YW\r
+ pBbBZGU4OJQkeB8/AmoULEpNT61Iy8wpQUgzcXCCDOcBGv4CpIa3uCAxtzgzHSJ/ilFRCmg0\r
+ SEIAJJFRmgfXC0tGrxjFgV4R5n0PUsUDTGRw3a+ABjMBDc5b/AlkcEkiQkqqgdH3Trh6iVcp\r
+ T2S4RchEYb19FzJs87ZktRQJbawRirF28Pm78eD9GafYUqzfPFx9uG6qwh3pH4uWOmzcmzCv\r
+ PafUtfXeFRuGG8+/ztwduDbJ13SavSfL7zd/av2Psy+q4T/6TIvDTsZd3rHlEO/M5rXuv+6k\r
+ ClhE8y3/wt7C47Rz2te3wbIW2UosxRmJhlrMRcWJAL5Ak2ohAwAA\r
+Cc: tomi.ollila@iki.fi, notmuch@notmuchmail.org\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+ <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Thu, 05 Jul 2012 18:58:31 -0000\r
+\r
+Quoth Mark Walters on Jul 05 at 9:37 am:\r
+> On Tue, 03 Jul 2012, Austin Clements <amdragon@MIT.EDU> wrote:\r
+> > The JSON format eliminates the complex escaping issues that have\r
+> > plagued the text search format. This uses the incremental JSON parser\r
+> > so that, like the text parser, it can output search results\r
+> > incrementally.\r
+> >\r
+> > This slows down the parser by about ~4X, but puts us in a good\r
+> > position to optimize either by improving the JSON parser (evidence\r
+> > suggests this can reduce the overhead to ~40% over the text format) or\r
+> > by switching to S-expressions (evidence suggests this will more than\r
+> > double performance over the text parser). [1]\r
+> >\r
+> > This also fixes the incremental search parsing test.\r
+> >\r
+> > [1] id:"20110720205007.GB21316@mit.edu"\r
+> > ---\r
+> > emacs/notmuch.el | 113 ++++++++++++++++++++++++++++++++----------------------\r
+> > test/emacs | 1 -\r
+> > 2 files changed, 67 insertions(+), 47 deletions(-)\r
+> >\r
+> > diff --git a/emacs/notmuch.el b/emacs/notmuch.el\r
+> > index 084cec6..2a09a98 100644\r
+> > --- a/emacs/notmuch.el\r
+> > +++ b/emacs/notmuch.el\r
+> > @@ -60,7 +60,7 @@\r
+> > (require 'notmuch-message)\r
+> > \r
+> > (defcustom notmuch-search-result-format\r
+> > - `(("date" . "%s ")\r
+> > + `(("date" . "%12s ")\r
+> > ("count" . "%-7s ")\r
+> > ("authors" . "%-20s ")\r
+> > ("subject" . "%s ")\r
+> > @@ -557,17 +557,14 @@ This function advances the next thread when finished."\r
+> > (notmuch-search-tag '("-inbox"))\r
+> > (notmuch-search-next-thread))\r
+> > \r
+> > -(defvar notmuch-search-process-filter-data nil\r
+> > - "Data that has not yet been processed.")\r
+> > -(make-variable-buffer-local 'notmuch-search-process-filter-data)\r
+> > -\r
+> > (defun notmuch-search-process-sentinel (proc msg)\r
+> > "Add a message to let user know when \"notmuch search\" exits"\r
+> > (let ((buffer (process-buffer proc))\r
+> > (status (process-status proc))\r
+> > (exit-status (process-exit-status proc))\r
+> > (never-found-target-thread nil))\r
+> > - (if (memq status '(exit signal))\r
+> > + (when (memq status '(exit signal))\r
+> > + (kill-buffer (process-get proc 'parse-buf))\r
+> > (if (buffer-live-p buffer)\r
+> > (with-current-buffer buffer\r
+> > (save-excursion\r
+> > @@ -577,8 +574,6 @@ This function advances the next thread when finished."\r
+> > (if (eq status 'signal)\r
+> > (insert "Incomplete search results (search process was killed).\n"))\r
+> > (when (eq status 'exit)\r
+> > - (if notmuch-search-process-filter-data\r
+> > - (insert (concat "Error: Unexpected output from notmuch search:\n" notmuch-search-process-filter-data)))\r
+> > (insert "End of search results.")\r
+> > (unless (= exit-status 0)\r
+> > (insert (format " (process returned %d)" exit-status)))\r
+> > @@ -757,45 +752,62 @@ non-authors is found, assume that all of the authors match."\r
+> > (insert (apply #'format string objects))\r
+> > (insert "\n")))\r
+> > \r
+> > +(defvar notmuch-search-process-state nil\r
+> > + "Parsing state of the search process filter.")\r
+> > +\r
+> > +(defvar notmuch-search-json-parser nil\r
+> > + "Incremental JSON parser for the search process filter.")\r
+> > +\r
+> > (defun notmuch-search-process-filter (proc string)\r
+> > "Process and filter the output of \"notmuch search\""\r
+> > - (let ((buffer (process-buffer proc)))\r
+> > - (if (buffer-live-p buffer)\r
+> > - (with-current-buffer buffer\r
+> > - (let ((line 0)\r
+> > - (more t)\r
+> > - (inhibit-read-only t)\r
+> > - (string (concat notmuch-search-process-filter-data string)))\r
+> > - (setq notmuch-search-process-filter-data nil)\r
+> > - (while more\r
+> > - (while (and (< line (length string)) (= (elt string line) ?\n))\r
+> > - (setq line (1+ line)))\r
+> > - (if (string-match "^thread:\\([0-9A-Fa-f]*\\) \\([^][]*\\) \\[\\([0-9]*\\)/\\([0-9]*\\)\\] \\([^;]*\\); \\(.*\\) (\\([^()]*\\))$" string line)\r
+> > - (let* ((thread-id (match-string 1 string))\r
+> > - (tags-str (match-string 7 string))\r
+> > - (result (list :thread thread-id\r
+> > - :date_relative (match-string 2 string)\r
+> > - :matched (string-to-number\r
+> > - (match-string 3 string))\r
+> > - :total (string-to-number\r
+> > - (match-string 4 string))\r
+> > - :authors (match-string 5 string)\r
+> > - :subject (match-string 6 string)\r
+> > - :tags (if tags-str\r
+> > - (save-match-data\r
+> > - (split-string tags-str))))))\r
+> > - (if (/= (match-beginning 0) line)\r
+> > - (notmuch-search-show-error\r
+> > - (substring string line (match-beginning 0))))\r
+> > - (notmuch-search-show-result result)\r
+> > - (set 'line (match-end 0)))\r
+> > - (set 'more nil)\r
+> > - (while (and (< line (length string)) (= (elt string line) ?\n))\r
+> > - (setq line (1+ line)))\r
+> > - (if (< line (length string))\r
+> > - (setq notmuch-search-process-filter-data (substring string line)))\r
+> > - ))))\r
+> > - (delete-process proc))))\r
+> > + (let ((results-buf (process-buffer proc))\r
+> > + (parse-buf (process-get proc 'parse-buf))\r
+> > + (inhibit-read-only t)\r
+> > + done)\r
+> > + (if (not (buffer-live-p results-buf))\r
+> > + (delete-process proc)\r
+> > + (with-current-buffer parse-buf\r
+> > + ;; Insert new data\r
+> > + (save-excursion\r
+> > + (goto-char (point-max))\r
+> > + (insert string)))\r
+> > + (with-current-buffer results-buf\r
+> > + (while (not done)\r
+> > + (condition-case nil\r
+> > + (case notmuch-search-process-state\r
+> > + ((begin)\r
+> > + ;; Enter the results list\r
+> > + (if (eq (notmuch-json-begin-compound\r
+> > + notmuch-search-json-parser) 'retry)\r
+> > + (setq done t)\r
+> > + (setq notmuch-search-process-state 'result)))\r
+> > + ((result)\r
+> > + ;; Parse a result\r
+> > + (let ((result (notmuch-json-read notmuch-search-json-parser)))\r
+> > + (case result\r
+> > + ((retry) (setq done t))\r
+> > + ((end) (setq notmuch-search-process-state 'end))\r
+> > + (otherwise (notmuch-search-show-result result)))))\r
+> > + ((end)\r
+> > + ;; Any trailing data is unexpected\r
+> > + (with-current-buffer parse-buf\r
+> > + (skip-chars-forward " \t\r\n")\r
+> > + (if (eobp)\r
+> > + (setq done t)\r
+> > + (signal 'json-error nil)))))\r
+> \r
+> This looks good to me. Would it make sense to put the "Any trailing\r
+> data" as a tiny function in notmuch-lib? something like \r
+> \r
+> (defun notmuch-json-assert-end-of-data ()\r
+> (skip-chars-forward " \t\r\n") \r
+> (if (eobp)\r
+> (setq done t) \r
+> (signal 'json-error nil)))\r
+\r
+Also a good idea.\r
+\r
+Thanks for the review! I'll be sending v2 shortly.\r
+\r
+> Best wishes\r
+> \r
+> Mark\r
projects / notmuch-archives.git / commitdiff