--- /dev/null
+Return-Path: <amdragon@mit.edu>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+ by olra.theworths.org (Postfix) with ESMTP id 4BF5A431FBC\r
+ for <notmuch@notmuchmail.org>; Sun, 1 Jul 2012 20:52:48 -0700 (PDT)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -0.7\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-0.7 tagged_above=-999 required=5\r
+ tests=[RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+ by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+ with ESMTP id 5zw4IQ-ybvPT for <notmuch@notmuchmail.org>;\r
+ Sun, 1 Jul 2012 20:52:47 -0700 (PDT)\r
+Received: from dmz-mailsec-scanner-4.mit.edu (DMZ-MAILSEC-SCANNER-4.MIT.EDU\r
+ [18.9.25.15])\r
+ by olra.theworths.org (Postfix) with ESMTP id 953E3431FAF\r
+ for <notmuch@notmuchmail.org>; Sun, 1 Jul 2012 20:52:47 -0700 (PDT)\r
+X-AuditID: 1209190f-b7f306d0000008b4-a8-4ff11b0d0601\r
+Received: from mailhub-auth-1.mit.edu ( [18.9.21.35])\r
+ by dmz-mailsec-scanner-4.mit.edu (Symantec Messaging Gateway) with SMTP\r
+ id C5.01.02228.D0B11FF4; Sun, 1 Jul 2012 23:52:46 -0400 (EDT)\r
+Received: from outgoing.mit.edu (OUTGOING-AUTH.MIT.EDU [18.7.22.103])\r
+ by mailhub-auth-1.mit.edu (8.13.8/8.9.2) with ESMTP id q623qjul011592; \r
+ Sun, 1 Jul 2012 23:52:45 -0400\r
+Received: from awakening.csail.mit.edu (awakening.csail.mit.edu [18.26.4.91])\r
+ (authenticated bits=0)\r
+ (User authenticated as amdragon@ATHENA.MIT.EDU)\r
+ by outgoing.mit.edu (8.13.6/8.12.4) with ESMTP id q623qgRi023541\r
+ (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NOT);\r
+ Sun, 1 Jul 2012 23:52:44 -0400 (EDT)\r
+Received: from amthrax by awakening.csail.mit.edu with local (Exim 4.77)\r
+ (envelope-from <amdragon@MIT.EDU>)\r
+ id 1SlXgn-0007kM-VK; Sun, 01 Jul 2012 23:52:42 -0400\r
+Date: Sun, 1 Jul 2012 23:52:41 -0400\r
+From: Austin Clements <amdragon@MIT.EDU>\r
+To: Tomi Ollila <tomi.ollila@iki.fi>\r
+Subject: Re: [PATCH] cli: notmuch-show with framing newlines between threads\r
+ in JSON.\r
+Message-ID: <20120702035241.GD6220@mit.edu>\r
+References: <1341041024-5342-1-git-send-email-markwalters1009@gmail.com>\r
+ <m24npr8701.fsf@guru.guru-group.fi> <20120702001215.GC6220@mit.edu>\r
+MIME-Version: 1.0\r
+Content-Type: text/plain; charset=us-ascii\r
+Content-Disposition: inline\r
+In-Reply-To: <20120702001215.GC6220@mit.edu>\r
+User-Agent: Mutt/1.5.21 (2010-09-15)\r
+X-Brightmail-Tracker:\r
+ H4sIAAAAAAAAA+NgFprAKsWRmVeSWpSXmKPExsUixCmqrMsn/dHfYONdRovrN2cyW7xZOY/V\r
+ gcnj8NeFLB7PVt1iDmCK4rJJSc3JLEst0rdL4Mr4t0KioFW6YmrfJtYGxmmiXYycHBICJhJz\r
+ Nl5ngrDFJC7cW8/WxcjFISSwj1Fi8ckFrBDOekaJ04+nMUI4J5gkZr/fBuUsYZTYfvoDO0g/\r
+ i4CKxPnG32Cz2AQ0JLbtX84IYosAxR+0rWcFsZkFpCW+/W4GqxEWCJc4vGkLG4jNK6AtsW3u\r
+ exaIoRMYJSa9nMkKkRCUODnzCQtEs5bEjX8vgZo5wAYt/8cBEuYU0JHY3rwR7AZRoF1TTm5j\r
+ m8AoNAtJ9ywk3bMQuhcwMq9ilE3JrdLNTczMKU5N1i1OTszLSy3SNdHLzSzRS00p3cQIDmxJ\r
+ /h2M3w4qHWIU4GBU4uFVuv3BX4g1say4MvcQoyQHk5Io70qJj/5CfEn5KZUZicUZ8UWlOanF\r
+ hxglOJiVRHifHn/vL8SbklhZlVqUD5OS5mBREue9mnLTX0ggPbEkNTs1tSC1CCYrw8GhJMEb\r
+ LQU0VLAoNT21Ii0zpwQhzcTBCTKcB2j4apAa3uKCxNzizHSI/ClGXY51b47cYBRiycvPS5US\r
+ 550FUiQAUpRRmgc3B5aQXjGKA70lzOsBUsUDTGZwk14BLWECWvJ89TuQJSWJCCmpBkbWoCOz\r
+ mc4khVzg6nbuYPLmejA3pJrbyS/WebJQ46mpl9RWfA7avVTQOl40PFD54VOD0Ef7k6TWrfvz\r
+ Qu2+bdZ/1mePQySnB35/tOlK5J17Tv5Ff03fWbS1iSbNCZtu6HhCfk6XXdIaD7XzffeTHh1q\r
+ Zip82GR20nBbl9SnfCE1gdKZRSvio5VYijMSDbWYi4oTARR0Lx8jAwAA\r
+Cc: notmuch@notmuchmail.org\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+ <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Mon, 02 Jul 2012 03:52:48 -0000\r
+\r
+Quoth myself on Jul 01 at 8:12 pm:\r
+> Quoth Tomi Ollila on Jul 02 at 1:13 am:\r
+> > On Sat, Jun 30 2012, Mark Walters <markwalters1009@gmail.com> wrote:\r
+> > \r
+> > > Add newlines between complete threads to make asynchronous parsing\r
+> > > of the JSON easier.\r
+> > > ---\r
+> > >\r
+> > > notmuch-pick uses the JSON output of notmuch show but, in many cases,\r
+> > > for many threads. This can take quite a long time when displaying a\r
+> > > large number of messages (say 20 seconds for the 10,000 messages in\r
+> > > the notmuch archive). Thus it is desirable to display results\r
+> > > incrementally in the same way that search currently does.\r
+> > >\r
+> > > To make this easier this patch adds newlines between each toplevel\r
+> > > thread. So the ouput becomes\r
+> > >\r
+> > > [\r
+> > > thread1\r
+> > > , thread2\r
+> > > , thread3\r
+> > > ...\r
+> > > , last_thread\r
+> > > ]\r
+> > >\r
+> > > Thus the parser can easily tell if it has enough data to do some more\r
+> > > parsing.\r
+> > >\r
+> > > Obviously, this changes the JSON output. This should not break any\r
+> > > consumer as the JSON parsers should not mind. However, it does break\r
+> > > several tests. Obviously, I will fix these but I wanted to check if\r
+> > > people were basically happy with the change first.\r
+> > \r
+> > To provide this feature rather than relying on newlines the parser should\r
+> > use it's state to notice when one thread ends. \r
+> > \r
+> > Such a change could be used (privately) for human consumption -- allowing \r
+> > free change of whitespace during inspection (in a debugging session or so).\r
+> > Computer software should not rely (or suffer) from any additional\r
+> > (or lack thereof) whitespace there is...\r
+> > \r
+> > ... or at least a really convicing argument for the chance needs to\r
+> > be presented (before "restricting" the json output notmuch spits out).\r
+> \r
+> Given a JSON parser that only knows how to parse complete JSON\r
+> expressions, it's potentially very inefficient to keep attempting to\r
+> parse something when you don't know if it's complete. The newlines\r
+> provide an in-band framing so the consumer knows when there's a\r
+> complete object to be parsed.\r
+> \r
+> In effect, this defines a super-protocol of JSON that's compatible\r
+> with standard JSON, but easy to incrementally parse.\r
+> \r
+> That said, just this weekend I implemented JSON-based search with\r
+> incremental JSON parsing and I took a slightly different approach. I\r
+> still put framing into the newlines of the search results, but rather\r
+> than rely on it for correctness, the consumer uses it as an\r
+> optimization that only hints that a complete JSON expression is\r
+> probably available. If the expression turns out to be incomplete,\r
+> that's okay.\r
+> \r
+> I considered building a fully-incremental JSON parser that never\r
+> backtracks by more than a token, which would eliminate even the cost\r
+> of reparsing, but if we do move to S-expressions (which I think we\r
+> should), we want to let Emacs' C implementation do as much of the\r
+> parsing as possible, and the only thing we can do with that is read a\r
+> complete expression.\r
+\r
+Actually, I take that back. While we can't do fast incremental\r
+S-expression parsing, `parse-partial-sexp' can tell us incrementally\r
+(and probably very quickly) *if* there's a complete expression ready\r
+to parse, so we could avoid calling into the parser at all unless it\r
+would succeed.\r
+\r
+I'll try this out in my incremental JSON parser and see how well it\r
+works.\r
projects / notmuch-archives.git / commitdiff