--- /dev/null
+Return-Path: <amdragon@mit.edu>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+ by olra.theworths.org (Postfix) with ESMTP id EAA7D431FCF\r
+ for <notmuch@notmuchmail.org>; Thu, 5 Jul 2012 13:52:44 -0700 (PDT)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -0.7\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-0.7 tagged_above=-999 required=5\r
+ tests=[RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+ by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+ with ESMTP id rwmQ05UoYS-g for <notmuch@notmuchmail.org>;\r
+ Thu, 5 Jul 2012 13:52:43 -0700 (PDT)\r
+Received: from dmz-mailsec-scanner-4.mit.edu (DMZ-MAILSEC-SCANNER-4.MIT.EDU\r
+ [18.9.25.15])\r
+ by olra.theworths.org (Postfix) with ESMTP id 16535431FAF\r
+ for <notmuch@notmuchmail.org>; Thu, 5 Jul 2012 13:52:43 -0700 (PDT)\r
+X-AuditID: 1209190f-b7f306d0000008b4-e8-4ff5fe9a0d29\r
+Received: from mailhub-auth-4.mit.edu ( [18.7.62.39])\r
+ by dmz-mailsec-scanner-4.mit.edu (Symantec Messaging Gateway) with SMTP\r
+ id 8C.18.02228.A9EF5FF4; Thu, 5 Jul 2012 16:52:42 -0400 (EDT)\r
+Received: from outgoing.mit.edu (OUTGOING-AUTH.MIT.EDU [18.7.22.103])\r
+ by mailhub-auth-4.mit.edu (8.13.8/8.9.2) with ESMTP id q65Kqg7P019979; \r
+ Thu, 5 Jul 2012 16:52:42 -0400\r
+Received: from drake.dyndns.org (26-4-182.dynamic.csail.mit.edu [18.26.4.182])\r
+ (authenticated bits=0)\r
+ (User authenticated as amdragon@ATHENA.MIT.EDU)\r
+ by outgoing.mit.edu (8.13.6/8.12.4) with ESMTP id q65KqbnJ027233\r
+ (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NOT);\r
+ Thu, 5 Jul 2012 16:52:39 -0400 (EDT)\r
+Received: from amthrax by drake.dyndns.org with local (Exim 4.77)\r
+ (envelope-from <amdragon@mit.edu>)\r
+ id 1Smt2T-0004XS-Fg; Thu, 05 Jul 2012 16:52:37 -0400\r
+From: Austin Clements <amdragon@MIT.EDU>\r
+To: notmuch@notmuchmail.org\r
+Subject: [PATCH v2 0/9] JSON-based search-mode\r
+Date: Thu, 5 Jul 2012 16:52:18 -0400\r
+Message-Id: <1341521547-15502-1-git-send-email-amdragon@mit.edu>\r
+X-Mailer: git-send-email 1.7.10\r
+In-Reply-To: <1341354059-29396-1-git-send-email-amdragon@mit.edu>\r
+References: <1341354059-29396-1-git-send-email-amdragon@mit.edu>\r
+X-Brightmail-Tracker:\r
+ H4sIAAAAAAAAA+NgFrrEIsWRmVeSWpSXmKPExsUixG6nrjvr31d/g2/nlCxWz+WxuH5zJrPF\r
+ m5XzWB2YPXbOusvucfjrQhaPZ6tuMQcwR3HZpKTmZJalFunbJXBl3OpZxFqw0qJiTo9lA+MP\r
+ nS5GTg4JAROJqR9nMkLYYhIX7q1n62Lk4hAS2McocfHCGlYIZz2jxKHuBVDOSSaJby++M0E4\r
+ cxkl5t3YzQLSzyagIbFt/3KwWSIC0hI7785mBbGZBQwlLn/tAKsRFtCTeL3zE1gNi4CqxPXW\r
+ BjCbV8BB4tflX+wQd8hLPL3fxwZicwo4Sqy4eRasVwioZtuHDtYJjPwLGBlWMcqm5Fbp5iZm\r
+ 5hSnJusWJyfm5aUW6Zro5WaW6KWmlG5iBAUXpyT/DsZvB5UOMQpwMCrx8BrmfvEXYk0sK67M\r
+ PcQoycGkJMrb+PurvxBfUn5KZUZicUZ8UWlOavEhRgkOZiUR3t4MoBxvSmJlVWpRPkxKmoNF\r
+ SZz3aspNfyGB9MSS1OzU1ILUIpisDAeHkgSvOjCKhASLUtNTK9Iyc0oQ0kwcnCDDeYCGa4DU\r
+ 8BYXJOYWZ6ZD5E8x6nKse3PkBqMQS15+XqqUOK80SJEASFFGaR7cHFhSeMUoDvSWMO+nv0BV\r
+ PMCEAjfpFdASJqAleYs/gSwpSURISTUwSja+PeFdPnk1q5LVb6nVDzh3n1mkkuf5ZXd69L/7\r
+ 6RU9mTFvr3ZN0OLhKU5wSj6omLNg2U62k+0uFcU555ONsq1fzNwf89frxnOH3+/D5SfIhqhX\r
+ MddMsqjbvWfh7ZsXEr9l7wuYddPV9NIswRfXeRq99K89KeH4eLpwZuQy/lX2xYW+c65oKrEU\r
+ ZyQaajEXFScCAE+NFQDlAgAA\r
+Cc: tomi.ollila@iki.fi\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+ <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Thu, 05 Jul 2012 20:52:45 -0000\r
+\r
+This should account for all of Mark's and Tomi's comments. This\r
+version\r
+* renames the "format" variables to "format-string" and "spec" to be\r
+ less confusing,\r
+* reverts to the original behavior of ignoring the user's format\r
+ specification for tags (since we make assumptions about this format\r
+ elsewhere),\r
+* swaps the error helper and search-target patches to fix the\r
+ temporary issue with error message placement,\r
+* adds documentation on point movement in the JSON parser,\r
+* breaks out the JSON EOF testing function,\r
+* beefs up a few commit messages,\r
+* and adds a NEWS patch.\r
+\r
+For ease of reviewing, the diff diff is below.\r
+\r
+I've written most of a follow-on patch series that cleans up the\r
+handling of metadata and tag changes by attaching the JSON result\r
+object to the result. This means we don't need a proliferation of\r
+text properties to store the result metadata, and we can make updates\r
+to a result (e.g., tag changes) by updating this result object and\r
+then re-rendering the result line from scratch, rather than trying to\r
+update the result line in place. This makes it possible to obey user\r
+formatting for the tag list and has other perks like recoloring\r
+results when their tags change. I'll send it along once this patch\r
+series is accepted.\r
+\r
+diff --git a/NEWS b/NEWS\r
+index d29ec5b..a1a6e93 100644\r
+--- a/NEWS\r
++++ b/NEWS\r
+@@ -14,6 +14,23 @@ Maildir tag synchronization\r
+ messages (typically causing new messages to not receive the "unread"\r
+ tag).\r
+ \r
++Emacs Interface\r
++---------------\r
++\r
++Search now uses the JSON format internally\r
++\r
++ This should address problems with unusual characters in authors and\r
++ subject lines that could confuse the old text-based search parser.\r
++\r
++The date shown in search results is no longer padded before applying\r
++user-specified formatting\r
++\r
++ Previously, the date in the search results was padded to fixed width\r
++ before being formatted with `notmuch-search-result-format`. It is\r
++ no longer padded. The default format has been updated, but if\r
++ you've customized this variable, you may have to change your date\r
++ format from `"%s "` to `"%12s "`.\r
++\r
+ Notmuch 0.13.2 (2012-06-02)\r
+ ===========================\r
+ \r
+diff --git a/emacs/notmuch-lib.el b/emacs/notmuch-lib.el\r
+index f7cda33..9e04d97 100644\r
+--- a/emacs/notmuch-lib.el\r
++++ b/emacs/notmuch-lib.el\r
+@@ -399,8 +399,9 @@ resynchronize after an error by moving point."\r
+ \r
+ Returns 'retry if there is insufficient input to parse the\r
+ beginning of the compound. If this is able to parse the\r
+-beginning of a compound, it returns t and later calls to\r
+-`notmuch-json-read' will return the compound's elements.\r
++beginning of a compound, it moves point past the token that opens\r
++the compound and returns t. Later calls to `notmuch-json-read'\r
++will return the compound's elements.\r
+ \r
+ Entering JSON objects is current unimplemented."\r
+ \r
+@@ -423,7 +424,8 @@ Entering JSON objects is current unimplemented."\r
+ Returns 'retry if there is insufficient input to parse a complete\r
+ JSON value. If the parser is currently inside a compound value\r
+ and the next token ends the list or object, returns 'end.\r
+-Otherwise, returns the value."\r
++Otherwise, moves point to just past the end of the value and\r
++returns the value."\r
+ \r
+ (with-current-buffer (notmuch-json-buffer jp)\r
+ (or\r
+@@ -474,11 +476,22 @@ Otherwise, returns the value."\r
+ (notmuch-json-partial-pos jp) nil\r
+ (notmuch-json-partial-state jp) nil)\r
+ ;; Parse the value\r
+- (let* ((json-object-type 'plist)\r
+- (json-array-type 'list)\r
+- (json-false nil))\r
++ (let ((json-object-type 'plist)\r
++ (json-array-type 'list)\r
++ (json-false nil))\r
+ (json-read)))))))\r
+ \r
++(defun notmuch-json-eof (jp)\r
++ "Signal a json-error if there is more input in JP's buffer.\r
++\r
++Moves point to the beginning of any trailing garbage or to the\r
++end of the buffer if there is no trailing garbage."\r
++\r
++ (with-current-buffer (notmuch-json-buffer jp)\r
++ (skip-chars-forward " \t\r\n")\r
++ (unless (eobp)\r
++ (signal 'json-error (list "Trailing garbage following JSON data")))))\r
++\r
+ (provide 'notmuch-lib)\r
+ \r
+ ;; Local Variables:\r
+diff --git a/emacs/notmuch.el b/emacs/notmuch.el\r
+index 2a09a98..fabb7c0 100644\r
+--- a/emacs/notmuch.el\r
++++ b/emacs/notmuch.el\r
+@@ -702,28 +702,29 @@ non-authors is found, assume that all of the authors match."\r
+ (overlay-put overlay 'isearch-open-invisible #'delete-overlay)))\r
+ (insert padding))))\r
+ \r
+-(defun notmuch-search-insert-field (field format result)\r
++(defun notmuch-search-insert-field (field format-string result)\r
+ (cond\r
+ ((string-equal field "date")\r
+- (insert (propertize (format format (plist-get result :date_relative))\r
++ (insert (propertize (format format-string (plist-get result :date_relative))\r
+ 'face 'notmuch-search-date)))\r
+ ((string-equal field "count")\r
+- (insert (propertize (format format (format "[%s/%s]"\r
+- (plist-get result :matched)\r
+- (plist-get result :total)))\r
++ (insert (propertize (format format-string\r
++ (format "[%s/%s]" (plist-get result :matched)\r
++ (plist-get result :total)))\r
+ 'face 'notmuch-search-count)))\r
+ ((string-equal field "subject")\r
+- (insert (propertize (format format (plist-get result :subject))\r
++ (insert (propertize (format format-string (plist-get result :subject))\r
+ 'face 'notmuch-search-subject)))\r
+ \r
+ ((string-equal field "authors")\r
+- (notmuch-search-insert-authors format (plist-get result :authors)))\r
++ (notmuch-search-insert-authors format-string (plist-get result :authors)))\r
+ \r
+ ((string-equal field "tags")\r
+- (insert\r
+- (format format (propertize\r
+- (mapconcat 'identity (plist-get result :tags) " ")\r
+- 'font-lock-face 'notmuch-tag-face))))))\r
++ ;; Ignore format-string here because notmuch-search-set-tags\r
++ ;; depends on the format of this\r
++ (insert (concat "(" (propertize\r
++ (mapconcat 'identity (plist-get result :tags) " ")\r
++ 'font-lock-face 'notmuch-tag-face) ")")))))\r
+ \r
+ (defun notmuch-search-show-result (result)\r
+ ;; Ignore excluded matches\r
+@@ -731,8 +732,8 @@ non-authors is found, assume that all of the authors match."\r
+ (let ((beg (point-max)))\r
+ (save-excursion\r
+ (goto-char beg)\r
+- (dolist (format notmuch-search-result-format)\r
+- (notmuch-search-insert-field (car format) (cdr format) result))\r
++ (dolist (spec notmuch-search-result-format)\r
++ (notmuch-search-insert-field (car spec) (cdr spec) result))\r
+ (insert "\n")\r
+ (notmuch-search-color-line beg (point) (plist-get result :tags))\r
+ (put-text-property beg (point) 'notmuch-search-thread-id\r
+@@ -790,11 +791,8 @@ non-authors is found, assume that all of the authors match."\r
+ (otherwise (notmuch-search-show-result result)))))\r
+ ((end)\r
+ ;; Any trailing data is unexpected\r
+- (with-current-buffer parse-buf\r
+- (skip-chars-forward " \t\r\n")\r
+- (if (eobp)\r
+- (setq done t)\r
+- (signal 'json-error nil)))))\r
++ (notmuch-json-eof notmuch-search-json-parser)\r
++ (setq done t)))\r
+ (json-error\r
+ ;; Do our best to resynchronize and ensure forward\r
+ ;; progress\r
+\r
+\r