1 Return-Path: <m.walters@qmul.ac.uk>
\r
2 X-Original-To: notmuch@notmuchmail.org
\r
3 Delivered-To: notmuch@notmuchmail.org
\r
4 Received: from localhost (localhost [127.0.0.1])
\r
5 by olra.theworths.org (Postfix) with ESMTP id 6A5E0431FBD
\r
6 for <notmuch@notmuchmail.org>; Sat, 1 Dec 2012 05:46:01 -0800 (PST)
\r
7 X-Virus-Scanned: Debian amavisd-new at olra.theworths.org
\r
11 X-Spam-Status: No, score=-1.098 tagged_above=-999 required=5
\r
12 tests=[DKIM_ADSP_CUSTOM_MED=0.001, FREEMAIL_FROM=0.001,
\r
13 NML_ADSP_CUSTOM_MED=1.2, RCVD_IN_DNSWL_MED=-2.3] autolearn=disabled
\r
14 Received: from olra.theworths.org ([127.0.0.1])
\r
15 by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)
\r
16 with ESMTP id c6LF9peg4S98 for <notmuch@notmuchmail.org>;
\r
17 Sat, 1 Dec 2012 05:45:57 -0800 (PST)
\r
18 Received: from mail2.qmul.ac.uk (mail2.qmul.ac.uk [138.37.6.6])
\r
19 (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
\r
20 (No client certificate requested)
\r
21 by olra.theworths.org (Postfix) with ESMTPS id C8E7A431FAF
\r
22 for <notmuch@notmuchmail.org>; Sat, 1 Dec 2012 05:45:56 -0800 (PST)
\r
23 Received: from smtp.qmul.ac.uk ([138.37.6.40])
\r
24 by mail2.qmul.ac.uk with esmtp (Exim 4.71)
\r
25 (envelope-from <m.walters@qmul.ac.uk>)
\r
26 id 1TenO9-0004bA-0T; Sat, 01 Dec 2012 13:45:51 +0000
\r
27 Received: from 93-97-24-31.zone5.bethere.co.uk ([93.97.24.31] helo=localhost)
\r
28 by smtp.qmul.ac.uk with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.69)
\r
29 (envelope-from <m.walters@qmul.ac.uk>)
\r
30 id 1TenO8-0001VV-8W; Sat, 01 Dec 2012 13:45:48 +0000
\r
31 From: Mark Walters <markwalters1009@gmail.com>
\r
32 To: Tomi Ollila <tomi.ollila@iki.fi>, Peter Feigl <craven@gmx.net>,
\r
33 notmuch@notmuchmail.org
\r
34 Subject: Re: [PATCH 1/3] Adding an S-expression structured output printer.
\r
35 In-Reply-To: <m2fw3pq4sf.fsf@guru.guru-group.fi>
\r
36 References: <1354264143-30173-1-git-send-email-craven@gmx.net>
\r
37 <871ufa9jpm.fsf@qmul.ac.uk> <m2txs6vwdy.fsf@guru.guru-group.fi>
\r
38 <87d2yu559y.fsf@qmul.ac.uk> <m2fw3pq4sf.fsf@guru.guru-group.fi>
\r
39 User-Agent: Notmuch/0.14+81~g9730584 (http://notmuchmail.org) Emacs/23.4.1
\r
40 (x86_64-pc-linux-gnu)
\r
41 Date: Sat, 01 Dec 2012 13:45:48 +0000
\r
42 Message-ID: <874nk5ansj.fsf@qmul.ac.uk>
\r
44 Content-Type: text/plain; charset=us-ascii
\r
45 X-Sender-Host-Address: 93.97.24.31
\r
46 X-QM-SPAM-Info: Sender has good ham record. :)
\r
47 X-QM-Body-MD5: a54945a81e0cdcffb39a6fbbe838342b (of first 20000 bytes)
\r
48 X-SpamAssassin-Score: -1.7
\r
49 X-SpamAssassin-SpamBar: -
\r
50 X-SpamAssassin-Report: The QM spam filters have analysed this message to
\r
52 spam. We require at least 5.0 points to mark a message as spam.
\r
53 This message scored -1.7 points.
\r
54 Summary of the scoring:
\r
55 * -2.3 RCVD_IN_DNSWL_MED RBL: Sender listed at http://www.dnswl.org/,
\r
57 * [138.37.6.40 listed in list.dnswl.org]
\r
58 * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail
\r
59 provider * (markwalters1009[at]gmail.com)
\r
60 * 0.6 AWL AWL: From: address is in the auto white-list
\r
61 X-QM-Scan-Virus: ClamAV says the message is clean
\r
62 X-BeenThere: notmuch@notmuchmail.org
\r
63 X-Mailman-Version: 2.1.13
\r
65 List-Id: "Use and development of the notmuch mail system."
\r
66 <notmuch.notmuchmail.org>
\r
67 List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,
\r
68 <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>
\r
69 List-Archive: <http://notmuchmail.org/pipermail/notmuch>
\r
70 List-Post: <mailto:notmuch@notmuchmail.org>
\r
71 List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>
\r
72 List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,
\r
73 <mailto:notmuch-request@notmuchmail.org?subject=subscribe>
\r
74 X-List-Received-Date: Sat, 01 Dec 2012 13:46:01 -0000
\r
76 On Sat, 01 Dec 2012, Tomi Ollila <tomi.ollila@iki.fi> wrote:
\r
77 > On Sat, Dec 01 2012, Mark Walters <markwalters1009@gmail.com> wrote:
\r
79 >> On Sat, 01 Dec 2012, Tomi Ollila <tomi.ollila@iki.fi> wrote:
\r
80 >>> On Sat, Dec 01 2012, Mark Walters wrote:
\r
84 >>>> Overall I like the series: I think I agree with all of Jani's
\r
87 >>>> My one extra comment is that I think we should decide on whether we also
\r
88 >>>> want a sexp plist version. I think we might want one for the emacs
\r
89 >>>> front-end as that currently uses plists for everything.
\r
91 >>>> If we do we might want to change the names a little, both for functions
\r
92 >>>> and options (eg sexp_a and sexp_p or something). Probably a lot of
\r
93 >>>> sprinter-sexp would be common to both versions.
\r
95 >>> This is an important question that needs to be addressed fast: options
\r
98 >>> 1) have options to spit both alist & plist formats
\r
99 >>> 2) when converting emacs to use s-expressions, convert it to use alists
\r
100 >>> 3) start using plists instead of alists in Peter's android client
\r
102 >> Ok I have looked at this and the changes needed to output plist (or
\r
103 >> both) are pretty small: the only functions from sprinter-sexp.c that
\r
104 >> need to be changed are sexp_end and sexp_map_key. The total diff from
\r
105 >> alist to plist is about 10 lines. I have a version which allows both
\r
106 >> (the same sprinter file creates both possibilities) and have hooked it
\r
107 >> into emacs/notmuch-show.el and it all seems to work.
\r
109 >> (Search is more difficult as that uses the async parser; indeed even for
\r
110 >> show I used sexp-at-point as suggested by Tomi which seems rather
\r
111 >> underdocumented but does seem to work)
\r
113 >> Given the ease with which we can allow both I think that would be my
\r
114 >> preference: the biggest problem is that slightly more cluttered option
\r
115 >> list (i.e., we have to allow both --format=sexpa and --format=sexpp or
\r
118 >> (I can post the patch doing the above but almost all of it is modifying the
\r
119 >> commands to choose alist or plist rather than modifying the
\r
120 >> sprinter-sexp itself)
\r
122 > As the diff is so small I agree that supporting 2 formats is good
\r
125 > In case this is done I suggest that we proceed the following way:
\r
127 > 1) Agree how we call these formats (sexpa & sexpp or something else)
\r
128 > 2) Peter does his updates, including to call the format as will be decided
\r
129 > 3) Mark posts his patches after Peter's work is pushed
\r
131 I think this seems a good approach. I should note that while the diff
\r
132 between alist and plist is only about 10 lines I do need to duplicate
\r
133 some of the surrounding code to allow both: its not huge but definitely
\r
134 more than 10 lines.
\r
136 I will clean it up a little and then post (as a diff on top of Peter's
\r
137 patches) just so people can judge if it looks basically acceptable.
\r
161 >>> In case (1) is chosen then we just need to support one more format.
\r
163 >>> How much work would it involve to convert emacs to receive content in
\r
164 >>> alists (and how feasible would that be)?
\r
166 >>> How much work would it require in Peter's client to use plists (and how
\r
167 >>> feasible would that be)?
\r
179 >>>> On Fri, 30 Nov 2012, Peter Feigl <craven@gmx.net> wrote:
\r
180 >>>>> This commit adds an sprinter for Lisp S-Expressions. Later commits will
\r
181 >>>>> use this printer.
\r
183 >>>>> The structure is the same as json, but:
\r
184 >>>>> - arrays are written as lists: ("foo" "bar" "baaz" 1 2 3)
\r
185 >>>>> - maps are written as a-lists: ((key "value") (other-key "other-value"))
\r
186 >>>>> - true is written as t
\r
187 >>>>> - false is written as nil
\r
188 >>>>> - null is written as nil
\r
190 >>>>> Makefile.local | 1 +
\r
191 >>>>> sprinter-sexp.c | 235 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
\r
192 >>>>> 2 files changed, 236 insertions(+)
\r
193 >>>>> create mode 100644 sprinter-sexp.c
\r
195 >>>>> diff --git a/Makefile.local b/Makefile.local
\r
196 >>>>> index 2b91946..0db1713 100644
\r
197 >>>>> --- a/Makefile.local
\r
198 >>>>> +++ b/Makefile.local
\r
199 >>>>> @@ -270,6 +270,7 @@ notmuch_client_srcs = \
\r
200 >>>>> notmuch-tag.c \
\r
201 >>>>> notmuch-time.c \
\r
202 >>>>> sprinter-json.c \
\r
203 >>>>> + sprinter-sexp.c \
\r
204 >>>>> sprinter-text.c \
\r
205 >>>>> query-string.c \
\r
206 >>>>> mime-node.c \
\r
207 >>>>> diff --git a/sprinter-sexp.c b/sprinter-sexp.c
\r
208 >>>>> new file mode 100644
\r
209 >>>>> index 0000000..8401c52
\r
210 >>>>> --- /dev/null
\r
211 >>>>> +++ b/sprinter-sexp.c
\r
212 >>>>> @@ -0,0 +1,235 @@
\r
213 >>>>> +#include <stdbool.h>
\r
214 >>>>> +#include <stdio.h>
\r
215 >>>>> +#include <talloc.h>
\r
216 >>>>> +#include "sprinter.h"
\r
218 >>>>> +struct sprinter_sexp {
\r
219 >>>>> + struct sprinter vtable;
\r
220 >>>>> + FILE *stream;
\r
221 >>>>> + /* Top of the state stack, or NULL if the printer is not currently
\r
222 >>>>> + * inside any aggregate types. */
\r
223 >>>>> + struct sexp_state *state;
\r
225 >>>>> + /* A flag to signify that a separator should be inserted in the
\r
226 >>>>> + * output as soon as possible.
\r
228 >>>>> + notmuch_bool_t insert_separator;
\r
231 >>>>> +struct sexp_state {
\r
232 >>>>> + struct sexp_state *parent;
\r
234 >>>>> + /* True if nothing has been printed in this aggregate yet.
\r
235 >>>>> + * Suppresses the space before a value. */
\r
236 >>>>> + notmuch_bool_t first;
\r
238 >>>>> + /* True if the state is a map state.
\r
239 >>>>> + Used to add a space between key/value pairs. */
\r
240 >>>>> + notmuch_bool_t in_map;
\r
242 >>>>> + /* The character that closes the current aggregate. */
\r
243 >>>>> + char close;
\r
246 >>>>> +/* Helper function to set up the stream to print a value. If this
\r
247 >>>>> + * value follows another value, prints a space. */
\r
248 >>>>> +static struct sprinter_sexp *
\r
249 >>>>> +sexp_begin_value (struct sprinter *sp)
\r
251 >>>>> + struct sprinter_sexp *sps = (struct sprinter_sexp *) sp;
\r
253 >>>>> + if (sps->state) {
\r
254 >>>>> + if (! sps->state->first) {
\r
255 >>>>> + if (sps->insert_separator) {
\r
256 >>>>> + fputc ('\n', sps->stream);
\r
257 >>>>> + sps->insert_separator = FALSE;
\r
259 >>>>> + if( ! sps->state->in_map)
\r
260 >>>>> + fputc (' ', sps->stream);
\r
263 >>>>> + sps->state->first = FALSE;
\r
266 >>>>> + return sps;
\r
269 >>>>> +/* Helper function to begin an aggregate type. Prints the open
\r
270 >>>>> + * character and pushes a new state frame. */
\r
272 >>>>> +sexp_begin_aggregate (struct sprinter *sp, char open, char close)
\r
274 >>>>> + struct sprinter_sexp *sps = sexp_begin_value (sp);
\r
275 >>>>> + struct sexp_state *state = talloc (sps, struct sexp_state);
\r
276 >>>>> + fputc (open, sps->stream);
\r
277 >>>>> + state->parent = sps->state;
\r
278 >>>>> + state->first = TRUE;
\r
279 >>>>> + state->in_map = FALSE;
\r
280 >>>>> + state->close = close;
\r
281 >>>>> + sps->state = state;
\r
285 >>>>> +sexp_begin_map (struct sprinter *sp)
\r
287 >>>>> + struct sprinter_sexp *sps = (struct sprinter_sexp *) sp;
\r
288 >>>>> + sexp_begin_aggregate (sp, '(', ')');
\r
289 >>>>> + sps->state->in_map = TRUE;
\r
293 >>>>> +sexp_begin_list (struct sprinter *sp)
\r
295 >>>>> + sexp_begin_aggregate (sp, '(', ')');
\r
299 >>>>> +sexp_end (struct sprinter *sp)
\r
301 >>>>> + struct sprinter_sexp *sps = (struct sprinter_sexp *) sp;
\r
302 >>>>> + struct sexp_state *state = sps->state;
\r
304 >>>>> + if (sps->state->in_map)
\r
305 >>>>> + fputc (')', sps->stream);
\r
306 >>>>> + fputc (sps->state->close, sps->stream);
\r
307 >>>>> + sps->state = state->parent;
\r
308 >>>>> + talloc_free (state);
\r
309 >>>>> + if (sps->state == NULL)
\r
310 >>>>> + fputc ('\n', sps->stream);
\r
313 >>>>> +/* This implementation supports embedded NULs as allowed by the JSON
\r
314 >>>>> + * specification and Unicode. Support for *parsing* embedded NULs
\r
315 >>>>> + * varies, but is generally not a problem outside of C-based parsers
\r
316 >>>>> + * (Python's json module and Emacs' json.el take embedded NULs in
\r
317 >>>>> + * stride). */
\r
319 >>>>> +sexp_string_len_internal (struct sprinter *sp, const char *val, size_t len, notmuch_bool_t quote)
\r
321 >>>>> + static const char *const escapes[] = {
\r
322 >>>>> + ['\"'] = "\\\"", ['\\'] = "\\\\", ['\b'] = "\\b",
\r
323 >>>>> + ['\f'] = "\\f", ['\n'] = "\\n", ['\t'] = "\\t"
\r
325 >>>>> + struct sprinter_sexp *sps = sexp_begin_value (sp);
\r
328 >>>>> + fputc ('"', sps->stream);
\r
329 >>>>> + for (; len; ++val, --len) {
\r
330 >>>>> + unsigned char ch = *val;
\r
331 >>>>> + if (ch < ARRAY_SIZE (escapes) && escapes[ch])
\r
332 >>>>> + fputs (escapes[ch], sps->stream);
\r
333 >>>>> + else if (ch >= 32)
\r
334 >>>>> + fputc (ch, sps->stream);
\r
336 >>>>> + fprintf (sps->stream, "\\u%04x", ch);
\r
339 >>>>> + fputc ('"', sps->stream);
\r
343 >>>>> +sexp_string_len (struct sprinter *sp, const char *val, size_t len)
\r
345 >>>>> + sexp_string_len_internal (sp, val, len, TRUE); /* print quoted */
\r
349 >>>>> +sexp_symbol_len (struct sprinter *sp, const char *val, size_t len)
\r
351 >>>>> + sexp_string_len_internal (sp, val, len, FALSE); /* print unquoted */
\r
355 >>>>> +sexp_string (struct sprinter *sp, const char *val)
\r
357 >>>>> + if (val == NULL)
\r
359 >>>>> + sexp_string_len (sp, val, strlen (val));
\r
363 >>>>> +sexp_symbol (struct sprinter *sp, const char *val)
\r
365 >>>>> + if (val == NULL)
\r
367 >>>>> + sexp_symbol_len (sp, val, strlen (val));
\r
371 >>>>> +sexp_integer (struct sprinter *sp, int val)
\r
373 >>>>> + struct sprinter_sexp *sps = sexp_begin_value (sp);
\r
375 >>>>> + fprintf (sps->stream, "%d", val);
\r
379 >>>>> +sexp_boolean (struct sprinter *sp, notmuch_bool_t val)
\r
381 >>>>> + struct sprinter_sexp *sps = sexp_begin_value (sp);
\r
383 >>>>> + fputs (val ? "t" : "nil", sps->stream);
\r
387 >>>>> +sexp_null (struct sprinter *sp)
\r
389 >>>>> + struct sprinter_sexp *sps = sexp_begin_value (sp);
\r
391 >>>>> + fputs ("nil", sps->stream);
\r
395 >>>>> +sexp_map_key (struct sprinter *sp, const char *key)
\r
397 >>>>> + struct sprinter_sexp *sps = (struct sprinter_sexp *) sp;
\r
399 >>>>> + if( sps->state->in_map && ! sps->state->first)
\r
400 >>>>> + fputs (") ", sps->stream);
\r
401 >>>>> + fputc ('(', sps->stream);
\r
402 >>>>> + sexp_symbol (sp, key);
\r
403 >>>>> + fputc (' ', sps->stream);
\r
407 >>>>> +sexp_set_prefix (unused (struct sprinter *sp), unused (const char *name))
\r
412 >>>>> +sexp_separator (struct sprinter *sp)
\r
414 >>>>> + struct sprinter_sexp *sps = (struct sprinter_sexp *) sp;
\r
416 >>>>> + sps->insert_separator = TRUE;
\r
419 >>>>> +struct sprinter *
\r
420 >>>>> +sprinter_sexp_create (const void *ctx, FILE *stream)
\r
422 >>>>> + static const struct sprinter_sexp template = {
\r
423 >>>>> + .vtable = {
\r
424 >>>>> + .begin_map = sexp_begin_map,
\r
425 >>>>> + .begin_list = sexp_begin_list,
\r
426 >>>>> + .end = sexp_end,
\r
427 >>>>> + .string = sexp_string,
\r
428 >>>>> + .string_len = sexp_string_len,
\r
429 >>>>> + .integer = sexp_integer,
\r
430 >>>>> + .boolean = sexp_boolean,
\r
431 >>>>> + .null = sexp_null,
\r
432 >>>>> + .map_key = sexp_map_key,
\r
433 >>>>> + .separator = sexp_separator,
\r
434 >>>>> + .set_prefix = sexp_set_prefix,
\r
435 >>>>> + .is_text_printer = FALSE,
\r
438 >>>>> + struct sprinter_sexp *res;
\r
440 >>>>> + res = talloc (ctx, struct sprinter_sexp);
\r
442 >>>>> + return NULL;
\r
444 >>>>> + *res = template;
\r
445 >>>>> + res->stream = stream;
\r
446 >>>>> + return &res->vtable;
\r
451 >>>>> _______________________________________________
\r
452 >>>>> notmuch mailing list
\r
453 >>>>> notmuch@notmuchmail.org
\r
454 >>>>> http://notmuchmail.org/mailman/listinfo/notmuch
\r
455 >>>> _______________________________________________
\r
456 >>>> notmuch mailing list
\r
457 >>>> notmuch@notmuchmail.org
\r
458 >>>> http://notmuchmail.org/mailman/listinfo/notmuch
\r
459 >> _______________________________________________
\r
460 >> notmuch mailing list
\r
461 >> notmuch@notmuchmail.org
\r
462 >> http://notmuchmail.org/mailman/listinfo/notmuch
\r