1 Return-Path: <amdragon@mit.edu>
\r
2 X-Original-To: notmuch@notmuchmail.org
\r
3 Delivered-To: notmuch@notmuchmail.org
\r
4 Received: from localhost (localhost [127.0.0.1])
\r
5 by olra.theworths.org (Postfix) with ESMTP id EA757431FBC
\r
6 for <notmuch@notmuchmail.org>; Sun, 1 Jul 2012 17:12:43 -0700 (PDT)
\r
7 X-Virus-Scanned: Debian amavisd-new at olra.theworths.org
\r
11 X-Spam-Status: No, score=-0.7 tagged_above=-999 required=5
\r
12 tests=[RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled
\r
13 Received: from olra.theworths.org ([127.0.0.1])
\r
14 by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)
\r
15 with ESMTP id MxFIJeuIzkrK for <notmuch@notmuchmail.org>;
\r
16 Sun, 1 Jul 2012 17:12:42 -0700 (PDT)
\r
17 Received: from dmz-mailsec-scanner-3.mit.edu (DMZ-MAILSEC-SCANNER-3.MIT.EDU
\r
19 by olra.theworths.org (Postfix) with ESMTP id 32899431FAF
\r
20 for <notmuch@notmuchmail.org>; Sun, 1 Jul 2012 17:12:42 -0700 (PDT)
\r
21 X-AuditID: 1209190e-b7fb56d0000008b2-61-4ff0e779327b
\r
22 Received: from mailhub-auth-3.mit.edu ( [18.9.21.43])
\r
23 by dmz-mailsec-scanner-3.mit.edu (Symantec Messaging Gateway) with SMTP
\r
24 id 0D.6B.02226.977E0FF4; Sun, 1 Jul 2012 20:12:41 -0400 (EDT)
\r
25 Received: from outgoing.mit.edu (OUTGOING-AUTH.MIT.EDU [18.7.22.103])
\r
26 by mailhub-auth-3.mit.edu (8.13.8/8.9.2) with ESMTP id q620CeXv017423;
\r
27 Sun, 1 Jul 2012 20:12:41 -0400
\r
28 Received: from awakening.csail.mit.edu (awakening.csail.mit.edu [18.26.4.91])
\r
29 (authenticated bits=0)
\r
30 (User authenticated as amdragon@ATHENA.MIT.EDU)
\r
31 by outgoing.mit.edu (8.13.6/8.12.4) with ESMTP id q620CaLp024059
\r
32 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NOT);
\r
33 Sun, 1 Jul 2012 20:12:39 -0400 (EDT)
\r
34 Received: from amthrax by awakening.csail.mit.edu with local (Exim 4.77)
\r
35 (envelope-from <amdragon@mit.edu>)
\r
36 id 1SlUFo-0007LF-50; Sun, 01 Jul 2012 20:12:36 -0400
\r
37 Date: Sun, 1 Jul 2012 20:12:35 -0400
\r
38 From: Austin Clements <amdragon@MIT.EDU>
\r
39 To: Tomi Ollila <tomi.ollila@iki.fi>
\r
40 Subject: Re: [PATCH] cli: notmuch-show with framing newlines between threads
\r
42 Message-ID: <20120702001215.GC6220@mit.edu>
\r
43 References: <1341041024-5342-1-git-send-email-markwalters1009@gmail.com>
\r
44 <m24npr8701.fsf@guru.guru-group.fi>
\r
46 Content-Type: text/plain; charset=us-ascii
\r
47 Content-Disposition: inline
\r
48 In-Reply-To: <m24npr8701.fsf@guru.guru-group.fi>
\r
49 User-Agent: Mutt/1.5.21 (2010-09-15)
\r
50 X-Brightmail-Tracker:
\r
51 H4sIAAAAAAAAA+NgFprOKsWRmVeSWpSXmKPExsUixCmqrVv5/IO/wZZmRYvVc3ksrt+cyWzx
\r
52 ZuU8Vgdmj52z7rJ7HP66kMXj2apbzAHMUVw2Kak5mWWpRfp2CVwZF7c+YS64IFtx6ddnpgbG
\r
53 TeJdjJwcEgImEnvW/mKGsMUkLtxbz9bFyMUhJLCPUaJ1Xyc7hLOeUWLuvglMEM4JJomJx4+z
\r
54 grQICSxhlFi3OqqLkYODRUBF4sz0BJAwm4CGxLb9yxlBbBGg8IO29WDlzAKuEjMu7GICsYUF
\r
55 wiUOb9rCBmLzCmhL3P3fywYxMk9i4renUHFBiZMzn7BA9GpJ3Pj3kglkFbOAtMTyfxwgYU4B
\r
56 A4kt71aDjRQFWjXl5Da2CYxCs5B0z0LSPQuhewEj8ypG2ZTcKt3cxMyc4tRk3eLkxLy81CJd
\r
57 Y73czBK91JTSTYzgMJfk28H49aDSIUYBDkYlHl7l2x/8hVgTy4orcw8xSnIwKYnyHn8MFOJL
\r
58 yk+pzEgszogvKs1JLT7EKMHBrCTC+/T4e38h3pTEyqrUonyYlDQHi5I475WUm/5CAumJJanZ
\r
59 qakFqUUwWRkODiUJ3lnPgIYKFqWmp1akZeaUIKSZODhBhvMADT8KUsNbXJCYW5yZDpE/xajL
\r
60 se7NkRuMQix5+XmpUuK8G0CKBECKMkrz4ObA0tMrRnGgt4R5J4FU8QBTG9ykV0BLmICWPF/9
\r
61 DmRJSSJCSqqBcZLB+Wkf3wpyT9+o+yL+fEZoZuCWM8ap1ZHzpn9bzDPDZEHF7oNWlx1UxT/k
\r
62 CO16V/CbY01/lZWv98nfk5RznAslctYszy5l16yv3vb+esTcnL1Fu2cc/ln4ScxrUpH52Ut1
\r
63 gqopL2rWy3nFXq2ZsH7OcU7ei7KyuxxV07Kcvp8s63own8mdR4mlOCPRUIu5qDgRAEZ2JCAq AwAA
\r
64 Cc: notmuch@notmuchmail.org
\r
65 X-BeenThere: notmuch@notmuchmail.org
\r
66 X-Mailman-Version: 2.1.13
\r
68 List-Id: "Use and development of the notmuch mail system."
\r
69 <notmuch.notmuchmail.org>
\r
70 List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,
\r
71 <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>
\r
72 List-Archive: <http://notmuchmail.org/pipermail/notmuch>
\r
73 List-Post: <mailto:notmuch@notmuchmail.org>
\r
74 List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>
\r
75 List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,
\r
76 <mailto:notmuch-request@notmuchmail.org?subject=subscribe>
\r
77 X-List-Received-Date: Mon, 02 Jul 2012 00:12:44 -0000
\r
79 Quoth Tomi Ollila on Jul 02 at 1:13 am:
\r
80 > On Sat, Jun 30 2012, Mark Walters <markwalters1009@gmail.com> wrote:
\r
82 > > Add newlines between complete threads to make asynchronous parsing
\r
83 > > of the JSON easier.
\r
86 > > notmuch-pick uses the JSON output of notmuch show but, in many cases,
\r
87 > > for many threads. This can take quite a long time when displaying a
\r
88 > > large number of messages (say 20 seconds for the 10,000 messages in
\r
89 > > the notmuch archive). Thus it is desirable to display results
\r
90 > > incrementally in the same way that search currently does.
\r
92 > > To make this easier this patch adds newlines between each toplevel
\r
93 > > thread. So the ouput becomes
\r
103 > > Thus the parser can easily tell if it has enough data to do some more
\r
106 > > Obviously, this changes the JSON output. This should not break any
\r
107 > > consumer as the JSON parsers should not mind. However, it does break
\r
108 > > several tests. Obviously, I will fix these but I wanted to check if
\r
109 > > people were basically happy with the change first.
\r
111 > To provide this feature rather than relying on newlines the parser should
\r
112 > use it's state to notice when one thread ends.
\r
114 > Such a change could be used (privately) for human consumption -- allowing
\r
115 > free change of whitespace during inspection (in a debugging session or so).
\r
116 > Computer software should not rely (or suffer) from any additional
\r
117 > (or lack thereof) whitespace there is...
\r
119 > ... or at least a really convicing argument for the chance needs to
\r
120 > be presented (before "restricting" the json output notmuch spits out).
\r
122 Given a JSON parser that only knows how to parse complete JSON
\r
123 expressions, it's potentially very inefficient to keep attempting to
\r
124 parse something when you don't know if it's complete. The newlines
\r
125 provide an in-band framing so the consumer knows when there's a
\r
126 complete object to be parsed.
\r
128 In effect, this defines a super-protocol of JSON that's compatible
\r
129 with standard JSON, but easy to incrementally parse.
\r
131 That said, just this weekend I implemented JSON-based search with
\r
132 incremental JSON parsing and I took a slightly different approach. I
\r
133 still put framing into the newlines of the search results, but rather
\r
134 than rely on it for correctness, the consumer uses it as an
\r
135 optimization that only hints that a complete JSON expression is
\r
136 probably available. If the expression turns out to be incomplete,
\r
139 I considered building a fully-incremental JSON parser that never
\r
140 backtracks by more than a token, which would eliminate even the cost
\r
141 of reparsing, but if we do move to S-expressions (which I think we
\r
142 should), we want to let Emacs' C implementation do as much of the
\r
143 parsing as possible, and the only thing we can do with that is read a
\r
144 complete expression.
\r
146 > Btw: AFAIC (json-read) parses the whole json object (ignoring whitespace,
\r
147 > including newlines outside strings). So I quess notmuch-pick uses something
\r
148 > slightly different (probably using json.el subroutines)..
\r
150 > Btw2: I'm very interested to see notmuch-pick in action -- I just don't
\r
151 > see this a way to do this particular support properly.
\r
153 > Btw3: is search is ever going to use json we'll face the same problem --
\r
154 > unless writing each line as a separate json object (and starting to use
\r
155 > s-expressions for speed)
\r
157 Done. I'll post the patches after a little more cleanup.
\r
159 > > Also, should devel/schemata be updated? It seems a little unclear as
\r
160 > > this is not really a "JSON" change as the JSON does not care about the
\r
165 > and best luck with your notmuch-pick work.
\r
projects / notmuch-archives.git / blob