From: Austin Clements Date: Thu, 5 Jul 2012 20:52:18 +0000 (+2000) Subject: [PATCH v2 0/9] JSON-based search-mode X-Git-Url: http://git.tremily.us/?p=notmuch-archives.git;a=commitdiff_plain;h=e9d6405f020c37fb729d5d5ba36bf70a79a834db [PATCH v2 0/9] JSON-based search-mode --- diff --git a/ea/3ef7ea3071ce64bbe1acee15fdc4595bb4fd60 b/ea/3ef7ea3071ce64bbe1acee15fdc4595bb4fd60 new file mode 100644 index 000000000..ed7b92c72 --- /dev/null +++ b/ea/3ef7ea3071ce64bbe1acee15fdc4595bb4fd60 @@ -0,0 +1,252 @@ +Return-Path: +X-Original-To: notmuch@notmuchmail.org +Delivered-To: notmuch@notmuchmail.org +Received: from localhost (localhost [127.0.0.1]) + by olra.theworths.org (Postfix) with ESMTP id EAA7D431FCF + for ; Thu, 5 Jul 2012 13:52:44 -0700 (PDT) +X-Virus-Scanned: Debian amavisd-new at olra.theworths.org +X-Spam-Flag: NO +X-Spam-Score: -0.7 +X-Spam-Level: +X-Spam-Status: No, score=-0.7 tagged_above=-999 required=5 + tests=[RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled +Received: from olra.theworths.org ([127.0.0.1]) + by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) + with ESMTP id rwmQ05UoYS-g for ; + Thu, 5 Jul 2012 13:52:43 -0700 (PDT) +Received: from dmz-mailsec-scanner-4.mit.edu (DMZ-MAILSEC-SCANNER-4.MIT.EDU + [18.9.25.15]) + by olra.theworths.org (Postfix) with ESMTP id 16535431FAF + for ; Thu, 5 Jul 2012 13:52:43 -0700 (PDT) +X-AuditID: 1209190f-b7f306d0000008b4-e8-4ff5fe9a0d29 +Received: from mailhub-auth-4.mit.edu ( [18.7.62.39]) + by dmz-mailsec-scanner-4.mit.edu (Symantec Messaging Gateway) with SMTP + id 8C.18.02228.A9EF5FF4; Thu, 5 Jul 2012 16:52:42 -0400 (EDT) +Received: from outgoing.mit.edu (OUTGOING-AUTH.MIT.EDU [18.7.22.103]) + by mailhub-auth-4.mit.edu (8.13.8/8.9.2) with ESMTP id q65Kqg7P019979; + Thu, 5 Jul 2012 16:52:42 -0400 +Received: from drake.dyndns.org (26-4-182.dynamic.csail.mit.edu [18.26.4.182]) + (authenticated bits=0) + (User authenticated as amdragon@ATHENA.MIT.EDU) + by outgoing.mit.edu (8.13.6/8.12.4) with ESMTP id q65KqbnJ027233 + (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NOT); + Thu, 5 Jul 2012 16:52:39 -0400 (EDT) +Received: from amthrax by drake.dyndns.org with local (Exim 4.77) + (envelope-from ) + id 1Smt2T-0004XS-Fg; Thu, 05 Jul 2012 16:52:37 -0400 +From: Austin Clements +To: notmuch@notmuchmail.org +Subject: [PATCH v2 0/9] JSON-based search-mode +Date: Thu, 5 Jul 2012 16:52:18 -0400 +Message-Id: <1341521547-15502-1-git-send-email-amdragon@mit.edu> +X-Mailer: git-send-email 1.7.10 +In-Reply-To: <1341354059-29396-1-git-send-email-amdragon@mit.edu> +References: <1341354059-29396-1-git-send-email-amdragon@mit.edu> +X-Brightmail-Tracker: + H4sIAAAAAAAAA+NgFrrEIsWRmVeSWpSXmKPExsUixG6nrjvr31d/g2/nlCxWz+WxuH5zJrPF + m5XzWB2YPXbOusvucfjrQhaPZ6tuMQcwR3HZpKTmZJalFunbJXBl3OpZxFqw0qJiTo9lA+MP + nS5GTg4JAROJqR9nMkLYYhIX7q1n62Lk4hAS2McocfHCGlYIZz2jxKHuBVDOSSaJby++M0E4 + cxkl5t3YzQLSzyagIbFt/3KwWSIC0hI7785mBbGZBQwlLn/tAKsRFtCTeL3zE1gNi4CqxPXW + BjCbV8BB4tflX+wQd8hLPL3fxwZicwo4Sqy4eRasVwioZtuHDtYJjPwLGBlWMcqm5Fbp5iZm + 5hSnJusWJyfm5aUW6Zro5WaW6KWmlG5iBAUXpyT/DsZvB5UOMQpwMCrx8BrmfvEXYk0sK67M + PcQoycGkJMrb+PurvxBfUn5KZUZicUZ8UWlOavEhRgkOZiUR3t4MoBxvSmJlVWpRPkxKmoNF + SZz3aspNfyGB9MSS1OzU1ILUIpisDAeHkgSvOjCKhASLUtNTK9Iyc0oQ0kwcnCDDeYCGa4DU + 8BYXJOYWZ6ZD5E8x6nKse3PkBqMQS15+XqqUOK80SJEASFFGaR7cHFhSeMUoDvSWMO+nv0BV + PMCEAjfpFdASJqAleYs/gSwpSURISTUwSja+PeFdPnk1q5LVb6nVDzh3n1mkkuf5ZXd69L/7 + 6RU9mTFvr3ZN0OLhKU5wSj6omLNg2U62k+0uFcU555ONsq1fzNwf89frxnOH3+/D5SfIhqhX + MddMsqjbvWfh7ZsXEr9l7wuYddPV9NIswRfXeRq99K89KeH4eLpwZuQy/lX2xYW+c65oKrEU + ZyQaajEXFScCAE+NFQDlAgAA +Cc: tomi.ollila@iki.fi +X-BeenThere: notmuch@notmuchmail.org +X-Mailman-Version: 2.1.13 +Precedence: list +List-Id: "Use and development of the notmuch mail system." + +List-Unsubscribe: , + +List-Archive: +List-Post: +List-Help: +List-Subscribe: , + +X-List-Received-Date: Thu, 05 Jul 2012 20:52:45 -0000 + +This should account for all of Mark's and Tomi's comments. This +version +* renames the "format" variables to "format-string" and "spec" to be + less confusing, +* reverts to the original behavior of ignoring the user's format + specification for tags (since we make assumptions about this format + elsewhere), +* swaps the error helper and search-target patches to fix the + temporary issue with error message placement, +* adds documentation on point movement in the JSON parser, +* breaks out the JSON EOF testing function, +* beefs up a few commit messages, +* and adds a NEWS patch. + +For ease of reviewing, the diff diff is below. + +I've written most of a follow-on patch series that cleans up the +handling of metadata and tag changes by attaching the JSON result +object to the result. This means we don't need a proliferation of +text properties to store the result metadata, and we can make updates +to a result (e.g., tag changes) by updating this result object and +then re-rendering the result line from scratch, rather than trying to +update the result line in place. This makes it possible to obey user +formatting for the tag list and has other perks like recoloring +results when their tags change. I'll send it along once this patch +series is accepted. + +diff --git a/NEWS b/NEWS +index d29ec5b..a1a6e93 100644 +--- a/NEWS ++++ b/NEWS +@@ -14,6 +14,23 @@ Maildir tag synchronization + messages (typically causing new messages to not receive the "unread" + tag). + ++Emacs Interface ++--------------- ++ ++Search now uses the JSON format internally ++ ++ This should address problems with unusual characters in authors and ++ subject lines that could confuse the old text-based search parser. ++ ++The date shown in search results is no longer padded before applying ++user-specified formatting ++ ++ Previously, the date in the search results was padded to fixed width ++ before being formatted with `notmuch-search-result-format`. It is ++ no longer padded. The default format has been updated, but if ++ you've customized this variable, you may have to change your date ++ format from `"%s "` to `"%12s "`. ++ + Notmuch 0.13.2 (2012-06-02) + =========================== + +diff --git a/emacs/notmuch-lib.el b/emacs/notmuch-lib.el +index f7cda33..9e04d97 100644 +--- a/emacs/notmuch-lib.el ++++ b/emacs/notmuch-lib.el +@@ -399,8 +399,9 @@ resynchronize after an error by moving point." + + Returns 'retry if there is insufficient input to parse the + beginning of the compound. If this is able to parse the +-beginning of a compound, it returns t and later calls to +-`notmuch-json-read' will return the compound's elements. ++beginning of a compound, it moves point past the token that opens ++the compound and returns t. Later calls to `notmuch-json-read' ++will return the compound's elements. + + Entering JSON objects is current unimplemented." + +@@ -423,7 +424,8 @@ Entering JSON objects is current unimplemented." + Returns 'retry if there is insufficient input to parse a complete + JSON value. If the parser is currently inside a compound value + and the next token ends the list or object, returns 'end. +-Otherwise, returns the value." ++Otherwise, moves point to just past the end of the value and ++returns the value." + + (with-current-buffer (notmuch-json-buffer jp) + (or +@@ -474,11 +476,22 @@ Otherwise, returns the value." + (notmuch-json-partial-pos jp) nil + (notmuch-json-partial-state jp) nil) + ;; Parse the value +- (let* ((json-object-type 'plist) +- (json-array-type 'list) +- (json-false nil)) ++ (let ((json-object-type 'plist) ++ (json-array-type 'list) ++ (json-false nil)) + (json-read))))))) + ++(defun notmuch-json-eof (jp) ++ "Signal a json-error if there is more input in JP's buffer. ++ ++Moves point to the beginning of any trailing garbage or to the ++end of the buffer if there is no trailing garbage." ++ ++ (with-current-buffer (notmuch-json-buffer jp) ++ (skip-chars-forward " \t\r\n") ++ (unless (eobp) ++ (signal 'json-error (list "Trailing garbage following JSON data"))))) ++ + (provide 'notmuch-lib) + + ;; Local Variables: +diff --git a/emacs/notmuch.el b/emacs/notmuch.el +index 2a09a98..fabb7c0 100644 +--- a/emacs/notmuch.el ++++ b/emacs/notmuch.el +@@ -702,28 +702,29 @@ non-authors is found, assume that all of the authors match." + (overlay-put overlay 'isearch-open-invisible #'delete-overlay))) + (insert padding)))) + +-(defun notmuch-search-insert-field (field format result) ++(defun notmuch-search-insert-field (field format-string result) + (cond + ((string-equal field "date") +- (insert (propertize (format format (plist-get result :date_relative)) ++ (insert (propertize (format format-string (plist-get result :date_relative)) + 'face 'notmuch-search-date))) + ((string-equal field "count") +- (insert (propertize (format format (format "[%s/%s]" +- (plist-get result :matched) +- (plist-get result :total))) ++ (insert (propertize (format format-string ++ (format "[%s/%s]" (plist-get result :matched) ++ (plist-get result :total))) + 'face 'notmuch-search-count))) + ((string-equal field "subject") +- (insert (propertize (format format (plist-get result :subject)) ++ (insert (propertize (format format-string (plist-get result :subject)) + 'face 'notmuch-search-subject))) + + ((string-equal field "authors") +- (notmuch-search-insert-authors format (plist-get result :authors))) ++ (notmuch-search-insert-authors format-string (plist-get result :authors))) + + ((string-equal field "tags") +- (insert +- (format format (propertize +- (mapconcat 'identity (plist-get result :tags) " ") +- 'font-lock-face 'notmuch-tag-face)))))) ++ ;; Ignore format-string here because notmuch-search-set-tags ++ ;; depends on the format of this ++ (insert (concat "(" (propertize ++ (mapconcat 'identity (plist-get result :tags) " ") ++ 'font-lock-face 'notmuch-tag-face) ")"))))) + + (defun notmuch-search-show-result (result) + ;; Ignore excluded matches +@@ -731,8 +732,8 @@ non-authors is found, assume that all of the authors match." + (let ((beg (point-max))) + (save-excursion + (goto-char beg) +- (dolist (format notmuch-search-result-format) +- (notmuch-search-insert-field (car format) (cdr format) result)) ++ (dolist (spec notmuch-search-result-format) ++ (notmuch-search-insert-field (car spec) (cdr spec) result)) + (insert "\n") + (notmuch-search-color-line beg (point) (plist-get result :tags)) + (put-text-property beg (point) 'notmuch-search-thread-id +@@ -790,11 +791,8 @@ non-authors is found, assume that all of the authors match." + (otherwise (notmuch-search-show-result result))))) + ((end) + ;; Any trailing data is unexpected +- (with-current-buffer parse-buf +- (skip-chars-forward " \t\r\n") +- (if (eobp) +- (setq done t) +- (signal 'json-error nil))))) ++ (notmuch-json-eof notmuch-search-json-parser) ++ (setq done t))) + (json-error + ;; Do our best to resynchronize and ensure forward + ;; progress + +