From: Tomi Ollila <tomi.ollila@iki.fi>
To: Mark Walters <markwalters1009@gmail.com>, notmuch@notmuchmail.org
Subject: Re: [PATCH] cli: notmuch-show with framing newlines between threads
	in JSON.
In-Reply-To: <1341041024-5342-1-git-send-email-markwalters1009@gmail.com>
References: <1341041024-5342-1-git-send-email-markwalters1009@gmail.com>
User-Agent: Notmuch/0.13.2+7~g2fb5bbc (http://notmuchmail.org) Emacs/23.1.1
	(x86_64-redhat-linux-gnu)
Date: Mon, 02 Jul 2012 01:13:34 +0300
Message-ID: <m24npr8701.fsf@guru.guru-group.fi>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Precedence: list

On Sat, Jun 30 2012, Mark Walters <markwalters1009@gmail.com> wrote:

> Add newlines between complete threads to make asynchronous parsing
> of the JSON easier.
> ---
>
> notmuch-pick uses the JSON output of notmuch show but, in many cases,
> for many threads. This can take quite a long time when displaying a
> large number of messages (say 20 seconds for the 10,000 messages in
> the notmuch archive). Thus it is desirable to display results
> incrementally in the same way that search currently does.
>
> To make this easier this patch adds newlines between each toplevel
> thread. So the ouput becomes
>
> [
> thread1
> , thread2
> , thread3
> ...
> , last_thread
> ]
>
> Thus the parser can easily tell if it has enough data to do some more
> parsing.
>
> Obviously, this changes the JSON output. This should not break any
> consumer as the JSON parsers should not mind. However, it does break
> several tests. Obviously, I will fix these but I wanted to check if
> people were basically happy with the change first.

To provide this feature rather than relying on newlines the parser should
use it's state to notice when one thread ends. 

Such a change could be used (privately) for human consumption -- allowing 
free change of whitespace during inspection (in a debugging session or so).
Computer software should not rely (or suffer) from any additional
(or lack thereof) whitespace there is...

... or at least a really convicing argument for the chance needs to
be presented (before "restricting" the json output notmuch spits out).

Btw: AFAIC (json-read) parses the whole json object (ignoring whitespace,
including newlines outside strings). So I quess notmuch-pick uses something
slightly different (probably using json.el subroutines)..

Btw2: I'm very interested to see notmuch-pick in action -- I just don't
see this a way to do this particular support properly.

Btw3: is search is ever going to use json we'll face the same problem -- 
unless writing each line as a separate json object (and starting to use 
s-expressions for speed)

> Also, should devel/schemata be updated? It seems a little unclear as
> this is not really a "JSON" change as the JSON does not care about the
> newlines.
>
> Best wishes

and best luck with your notmuch-pick work.

>
> Mark

Tomi

>
>
>  notmuch-show.c |    5 +++++
>  1 files changed, 5 insertions(+), 0 deletions(-)
>
> diff --git a/notmuch-show.c b/notmuch-show.c
> index 195e318..4a1d699 100644
> --- a/notmuch-show.c
> +++ b/notmuch-show.c
> @@ -942,6 +942,8 @@ do_show (void *ctx,
>  
>      if (format->message_set_start)
>  	fputs (format->message_set_start, stdout);
> +    if (format == &format_json)
> +	fputs ("\n", stdout);
>  
>      for (threads = notmuch_query_search_threads (query);
>  	 notmuch_threads_valid (threads);
> @@ -963,6 +965,9 @@ do_show (void *ctx,
>  	if (status && !res)
>  	    res = status;
>  
> +	if (format == &format_json)
> +	    fputs ("\n", stdout);
> +
>  	notmuch_thread_destroy (thread);
>  
>      }
> -- 
> 1.7.9.1
>
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch