--- /dev/null
+Return-Path: <m.walters@qmul.ac.uk>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+ by olra.theworths.org (Postfix) with ESMTP id DF808431FAE\r
+ for <notmuch@notmuchmail.org>; Thu, 5 Jul 2012 14:44:43 -0700 (PDT)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -1.098\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-1.098 tagged_above=-999 required=5\r
+ tests=[DKIM_ADSP_CUSTOM_MED=0.001, FREEMAIL_FROM=0.001,\r
+ NML_ADSP_CUSTOM_MED=1.2, RCVD_IN_DNSWL_MED=-2.3] autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+ by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+ with ESMTP id EOjfER5m45hl for <notmuch@notmuchmail.org>;\r
+ Thu, 5 Jul 2012 14:44:39 -0700 (PDT)\r
+Received: from mail2.qmul.ac.uk (mail2.qmul.ac.uk [138.37.6.6])\r
+ (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))\r
+ (No client certificate requested)\r
+ by olra.theworths.org (Postfix) with ESMTPS id 52CBF431FB6\r
+ for <notmuch@notmuchmail.org>; Thu, 5 Jul 2012 14:44:39 -0700 (PDT)\r
+Received: from smtp.qmul.ac.uk ([138.37.6.40])\r
+ by mail2.qmul.ac.uk with esmtp (Exim 4.71)\r
+ (envelope-from <m.walters@qmul.ac.uk>)\r
+ id 1Smtqk-0006Rh-9N; Thu, 05 Jul 2012 22:44:34 +0100\r
+Received: from 94-192-233-223.zone6.bethere.co.uk ([94.192.233.223]\r
+ helo=localhost)\r
+ by smtp.qmul.ac.uk with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.69)\r
+ (envelope-from <m.walters@qmul.ac.uk>)\r
+ id 1Smtqj-0005Kx-ID; Thu, 05 Jul 2012 22:44:33 +0100\r
+From: Mark Walters <markwalters1009@gmail.com>\r
+To: Austin Clements <amdragon@MIT.EDU>, notmuch@notmuchmail.org\r
+Subject: Re: [PATCH v2 0/9] JSON-based search-mode\r
+In-Reply-To: <1341521547-15502-1-git-send-email-amdragon@mit.edu>\r
+References: <1341354059-29396-1-git-send-email-amdragon@mit.edu>\r
+ <1341521547-15502-1-git-send-email-amdragon@mit.edu>\r
+User-Agent: Notmuch/0.13.2+76~g7e62702 (http://notmuchmail.org) Emacs/23.4.1\r
+ (x86_64-pc-linux-gnu)\r
+Date: Thu, 05 Jul 2012 22:44:30 +0100\r
+Message-ID: <87y5mx3mtd.fsf@qmul.ac.uk>\r
+MIME-Version: 1.0\r
+Content-Type: text/plain; charset=us-ascii\r
+X-Sender-Host-Address: 94.192.233.223\r
+X-QM-SPAM-Info: Sender has good ham record. :)\r
+X-QM-Body-MD5: bda545a15f7900316958d969bf22134f (of first 20000 bytes)\r
+X-SpamAssassin-Score: -1.8\r
+X-SpamAssassin-SpamBar: -\r
+X-SpamAssassin-Report: The QM spam filters have analysed this message to\r
+ determine if it is\r
+ spam. We require at least 5.0 points to mark a message as spam.\r
+ This message scored -1.8 points.\r
+ Summary of the scoring: \r
+ * -2.3 RCVD_IN_DNSWL_MED RBL: Sender listed at http://www.dnswl.org/,\r
+ * medium trust\r
+ * [138.37.6.40 listed in list.dnswl.org]\r
+ * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail\r
+ provider * (markwalters1009[at]gmail.com)\r
+ * -0.0 T_RP_MATCHES_RCVD Envelope sender domain matches handover relay\r
+ * domain\r
+ * 0.5 AWL AWL: From: address is in the auto white-list\r
+X-QM-Scan-Virus: ClamAV says the message is clean\r
+Cc: tomi.ollila@iki.fi\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+ <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Thu, 05 Jul 2012 21:44:44 -0000\r
+\r
+On Thu, 05 Jul 2012, Austin Clements <amdragon@MIT.EDU> wrote:\r
+> This should account for all of Mark's and Tomi's comments. This\r
+> version\r
+> * renames the "format" variables to "format-string" and "spec" to be\r
+> less confusing,\r
+> * reverts to the original behavior of ignoring the user's format\r
+> specification for tags (since we make assumptions about this format\r
+> elsewhere),\r
+> * swaps the error helper and search-target patches to fix the\r
+> temporary issue with error message placement,\r
+> * adds documentation on point movement in the JSON parser,\r
+> * breaks out the JSON EOF testing function,\r
+> * beefs up a few commit messages,\r
+> * and adds a NEWS patch.\r
+>\r
+> For ease of reviewing, the diff diff is below.\r
+>\r
+> I've written most of a follow-on patch series that cleans up the\r
+> handling of metadata and tag changes by attaching the JSON result\r
+> object to the result. This means we don't need a proliferation of\r
+> text properties to store the result metadata, and we can make updates\r
+> to a result (e.g., tag changes) by updating this result object and\r
+> then re-rendering the result line from scratch, rather than trying to\r
+> update the result line in place. This makes it possible to obey user\r
+> formatting for the tag list and has other perks like recoloring\r
+> results when their tags change. I'll send it along once this patch\r
+> series is accepted.\r
+>\r
+> diff --git a/NEWS b/NEWS\r
+> index d29ec5b..a1a6e93 100644\r
+> --- a/NEWS\r
+> +++ b/NEWS\r
+> @@ -14,6 +14,23 @@ Maildir tag synchronization\r
+> messages (typically causing new messages to not receive the "unread"\r
+> tag).\r
+> \r
+> +Emacs Interface\r
+> +---------------\r
+> +\r
+> +Search now uses the JSON format internally\r
+> +\r
+> + This should address problems with unusual characters in authors and\r
+> + subject lines that could confuse the old text-based search parser.\r
+> +\r
+> +The date shown in search results is no longer padded before applying\r
+> +user-specified formatting\r
+> +\r
+> + Previously, the date in the search results was padded to fixed width\r
+> + before being formatted with `notmuch-search-result-format`. It is\r
+> + no longer padded. The default format has been updated, but if\r
+> + you've customized this variable, you may have to change your date\r
+> + format from `"%s "` to `"%12s "`.\r
+> +\r
+> Notmuch 0.13.2 (2012-06-02)\r
+> ===========================\r
+> \r
+> diff --git a/emacs/notmuch-lib.el b/emacs/notmuch-lib.el\r
+> index f7cda33..9e04d97 100644\r
+> --- a/emacs/notmuch-lib.el\r
+> +++ b/emacs/notmuch-lib.el\r
+> @@ -399,8 +399,9 @@ resynchronize after an error by moving point."\r
+> \r
+> Returns 'retry if there is insufficient input to parse the\r
+> beginning of the compound. If this is able to parse the\r
+> -beginning of a compound, it returns t and later calls to\r
+> -`notmuch-json-read' will return the compound's elements.\r
+> +beginning of a compound, it moves point past the token that opens\r
+> +the compound and returns t. Later calls to `notmuch-json-read'\r
+> +will return the compound's elements.\r
+> \r
+> Entering JSON objects is current unimplemented."\r
+> \r
+> @@ -423,7 +424,8 @@ Entering JSON objects is current unimplemented."\r
+> Returns 'retry if there is insufficient input to parse a complete\r
+> JSON value. If the parser is currently inside a compound value\r
+> and the next token ends the list or object, returns 'end.\r
+> -Otherwise, returns the value."\r
+> +Otherwise, moves point to just past the end of the value and\r
+> +returns the value."\r
+\r
+I think that point can move when 'retry is returned (past terminators\r
+and commas for example). It might also be worth saying that it returns\r
+the next value passing command and terminators and whitespace after\r
+point.\r
+\r
+> \r
+> (with-current-buffer (notmuch-json-buffer jp)\r
+> (or\r
+> @@ -474,11 +476,22 @@ Otherwise, returns the value."\r
+> (notmuch-json-partial-pos jp) nil\r
+> (notmuch-json-partial-state jp) nil)\r
+> ;; Parse the value\r
+> - (let* ((json-object-type 'plist)\r
+> - (json-array-type 'list)\r
+> - (json-false nil))\r
+> + (let ((json-object-type 'plist)\r
+> + (json-array-type 'list)\r
+> + (json-false nil))\r
+> (json-read)))))))\r
+> \r
+> +(defun notmuch-json-eof (jp)\r
+> + "Signal a json-error if there is more input in JP's buffer.\r
+\r
+Would `data' be better than `input' (to distinguish allowed whitespace\r
+from disallowed data)?\r
+\r
+> +Moves point to the beginning of any trailing garbage or to the\r
+> +end of the buffer if there is no trailing garbage."\r
+> +\r
+> + (with-current-buffer (notmuch-json-buffer jp)\r
+> + (skip-chars-forward " \t\r\n")\r
+> + (unless (eobp)\r
+> + (signal 'json-error (list "Trailing garbage following JSON data")))))\r
+> +\r
+> (provide 'notmuch-lib)\r
+> \r
+> ;; Local Variables:\r
+> diff --git a/emacs/notmuch.el b/emacs/notmuch.el\r
+> index 2a09a98..fabb7c0 100644\r
+> --- a/emacs/notmuch.el\r
+> +++ b/emacs/notmuch.el\r
+> @@ -702,28 +702,29 @@ non-authors is found, assume that all of the authors match."\r
+> (overlay-put overlay 'isearch-open-invisible #'delete-overlay)))\r
+> (insert padding))))\r
+> \r
+> -(defun notmuch-search-insert-field (field format result)\r
+> +(defun notmuch-search-insert-field (field format-string result)\r
+> (cond\r
+> ((string-equal field "date")\r
+> - (insert (propertize (format format (plist-get result :date_relative))\r
+> + (insert (propertize (format format-string (plist-get result :date_relative))\r
+> 'face 'notmuch-search-date)))\r
+> ((string-equal field "count")\r
+> - (insert (propertize (format format (format "[%s/%s]"\r
+> - (plist-get result :matched)\r
+> - (plist-get result :total)))\r
+> + (insert (propertize (format format-string\r
+> + (format "[%s/%s]" (plist-get result :matched)\r
+> + (plist-get result :total)))\r
+> 'face 'notmuch-search-count)))\r
+> ((string-equal field "subject")\r
+> - (insert (propertize (format format (plist-get result :subject))\r
+> + (insert (propertize (format format-string (plist-get result :subject))\r
+> 'face 'notmuch-search-subject)))\r
+> \r
+> ((string-equal field "authors")\r
+> - (notmuch-search-insert-authors format (plist-get result :authors)))\r
+> + (notmuch-search-insert-authors format-string (plist-get result :authors)))\r
+> \r
+> ((string-equal field "tags")\r
+> - (insert\r
+> - (format format (propertize\r
+> - (mapconcat 'identity (plist-get result :tags) " ")\r
+> - 'font-lock-face 'notmuch-tag-face))))))\r
+> + ;; Ignore format-string here because notmuch-search-set-tags\r
+> + ;; depends on the format of this\r
+> + (insert (concat "(" (propertize\r
+> + (mapconcat 'identity (plist-get result :tags) " ")\r
+> + 'font-lock-face 'notmuch-tag-face) ")")))))\r
+> \r
+> (defun notmuch-search-show-result (result)\r
+> ;; Ignore excluded matches\r
+> @@ -731,8 +732,8 @@ non-authors is found, assume that all of the authors match."\r
+> (let ((beg (point-max)))\r
+> (save-excursion\r
+> (goto-char beg)\r
+> - (dolist (format notmuch-search-result-format)\r
+> - (notmuch-search-insert-field (car format) (cdr format) result))\r
+> + (dolist (spec notmuch-search-result-format)\r
+> + (notmuch-search-insert-field (car spec) (cdr spec) result))\r
+> (insert "\n")\r
+> (notmuch-search-color-line beg (point) (plist-get result :tags))\r
+> (put-text-property beg (point) 'notmuch-search-thread-id\r
+> @@ -790,11 +791,8 @@ non-authors is found, assume that all of the authors match."\r
+> (otherwise (notmuch-search-show-result result)))))\r
+> ((end)\r
+> ;; Any trailing data is unexpected\r
+> - (with-current-buffer parse-buf\r
+> - (skip-chars-forward " \t\r\n")\r
+> - (if (eobp)\r
+> - (setq done t)\r
+> - (signal 'json-error nil)))))\r
+> + (notmuch-json-eof notmuch-search-json-parser)\r
+> + (setq done t)))\r
+\r
+I think this used to only set `done' if there was not trailing data but\r
+now does so anyway?\r
+\r
+Best wishes\r
+\r
+Mark\r
+\r
+> (json-error\r
+> ;; Do our best to resynchronize and ensure forward\r
+> ;; progress\r