Return-Path: X-Original-To: notmuch@notmuchmail.org Delivered-To: notmuch@notmuchmail.org Received: from localhost (localhost [127.0.0.1]) by olra.theworths.org (Postfix) with ESMTP id 1877F431FBD for ; Wed, 8 Aug 2012 11:47:43 -0700 (PDT) X-Virus-Scanned: Debian amavisd-new at olra.theworths.org X-Spam-Flag: NO X-Spam-Score: 0 X-Spam-Level: X-Spam-Status: No, score=0 tagged_above=-999 required=5 tests=[none] autolearn=disabled Received: from olra.theworths.org ([127.0.0.1]) by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RJlYXhFsvrvo for ; Wed, 8 Aug 2012 11:47:41 -0700 (PDT) Received: from guru.guru-group.fi (guru.guru-group.fi [46.183.73.34]) by olra.theworths.org (Postfix) with ESMTP id 6B127431FAE for ; Wed, 8 Aug 2012 11:47:41 -0700 (PDT) Received: by guru.guru-group.fi (Postfix, from userid 501) id E59CD1002A4; Wed, 8 Aug 2012 21:47:49 +0300 (EEST) From: Tomi Ollila To: Austin Clements Subject: Re: [PATCH v2] test: Add test for messages with missing headers In-Reply-To: <20120808154608.GD11179@mit.edu> References: <1344374501-28185-1-git-send-email-amdragon@mit.edu> <1344389313-7886-1-git-send-email-amdragon@mit.edu> <20120808154608.GD11179@mit.edu> User-Agent: Notmuch/0.13.2+128~g07453d8 (http://notmuchmail.org) Emacs/23.1.1 (x86_64-redhat-linux-gnu) X-Face: HhBM'cA~ MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: notmuch@notmuchmail.org X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Aug 2012 18:47:43 -0000 On Wed, Aug 08 2012, Austin Clements wrote: > Quoth Tomi Ollila on Aug 08 at 11:08 am: >> On Wed, Aug 08 2012, Austin Clements wrote: >> >> > Currently the JSON tests for search and show are broken because >> > notmuch attempts to dereference a NULL pointer. >> > --- >> > This version fixes the "Show: text" test so that it sanitize its >> > output and doesn't hard-code my test paths. >> >> +1 >> >> Tomi >> >> w/ python json(.tool) the original order cannot be preserved as the parser >> stores content into dictionary -- without sorting those came in some >> internal python order... the following code could be used to use less > > To be fair, it can output them in original order, but doing so > requires Python 2.7: > > python -c 'import sys,json,collections; json.dump(json.load(sys.stdin, > object_pairs_hook=collections.OrderedDict), sys.stdout, indent=4)' ack. > >> indentation, though: >> >> python -c 'import sys,json; j = json.load(sys.stdin); >> json.dump(j, sys.stdout, sort_keys=True, indent=2)' < input_file >> >> The other "problem" with json.tool is that it converts non-ascii chars >> to \uNNNN values :/. > > This one doesn't require 2.7. > > python -c 'import sys,json,codecs; json.dump(json.load(sys.stdin), > codecs.getwriter("utf8")(sys.stdout), indent=4, ensure_ascii=False)' nice. > > Though I think that, for test canonicalization, \uNNNN is probably > less error/locale prone. true. > >> What we could do is to dig a simple c json formatter -- someday in the >> future, maybe -- but for *now* this is the best we can have :D > > Not another JSON parser/printer! Why not, one from scratch, without looking any source there is already >;) Well, you're right; the json.tool version python 2.6 provides is good -- -- ordering and those \uNNNN:s doesn't really matter. Probably thinking (or planning!) anything else at this time is waste of time. At least these 2 questions remain: Is the test requirement of python 2.6 suitable/acceptable ? How do we pretty-print S-expression syntax ;) ? Tomi > >> > >> > test/missing-headers | 162 ++++++++++++++++++++++++++++++++++++++++++++++++++ >> > test/notmuch-test | 1 + >> > 2 files changed, 163 insertions(+) >> > create mode 100755 test/missing-headers >> > >> > diff --git a/test/missing-headers b/test/missing-headers >> > new file mode 100755 >> > index 0000000..e79f922 >> > --- /dev/null >> > +++ b/test/missing-headers >> > @@ -0,0 +1,162 @@ >> > +#!/usr/bin/env bash >> > +test_description='messages with missing headers' >> > +. ./test-lib.sh >> > + >> > +# Notmuch requires at least one of from, subject, or to or it will >> > +# ignore the file. Generate two messages so that together they cover >> > +# all possible missing headers. We also give one of the messages a >> > +# date to ensure stable result ordering. >> > + >> > +cat < "${MAIL_DIR}/msg-2" >> > +To: Notmuch Test Suite >> > +Date: Fri, 05 Jan 2001 15:43:57 +0000 >> > + >> > +Body >> > +EOF >> > + >> > +cat < "${MAIL_DIR}/msg-1" >> > +From: Notmuch Test Suite >> > + >> > +Body >> > +EOF >> > + >> > +NOTMUCH_NEW >> > + >> > +test_begin_subtest "Search: text" >> > +output=$(notmuch search '*' | notmuch_search_sanitize) >> > +test_expect_equal "$output" "\ >> > +thread:XXX 2001-01-05 [1/1] (null); (inbox unread) >> > +thread:XXX 1970-01-01 [1/1] Notmuch Test Suite; (inbox unread)" >> > + >> > +test_begin_subtest "Search: json" >> > +test_subtest_known_broken >> > +output=$(notmuch search --format=json '*' | notmuch_search_sanitize) >> > +test_expect_equal_json "$output" ' >> > +[ >> > + { >> > + "authors": "", >> > + "date_relative": "2001-01-05", >> > + "matched": 1, >> > + "subject": "", >> > + "tags": [ >> > + "inbox", >> > + "unread" >> > + ], >> > + "thread": "XXX", >> > + "timestamp": 978709437, >> > + "total": 1 >> > + }, >> > + { >> > + "authors": "Notmuch Test Suite", >> > + "date_relative": "1970-01-01", >> > + "matched": 1, >> > + "subject": "", >> > + "tags": [ >> > + "inbox", >> > + "unread" >> > + ], >> > + "thread": "XXX", >> > + "timestamp": 0, >> > + "total": 1 >> > + } >> > +]' >> > + >> > +test_begin_subtest "Show: text" >> > +output=$(notmuch show '*' | notmuch_show_sanitize) >> > +test_expect_equal "$output" "\ >> > + message{ id:notmuch-sha1-7a6e4eac383ef958fcd3ebf2143db71b8ff01161 depth:0 match:1 excluded:0 filename:/XXX/mail/msg-2 >> > + header{ >> > + (2001-01-05) (inbox unread) >> > +Subject: (null) >> > +From: (null) >> > +To: Notmuch Test Suite >> > +Date: Fri, 05 Jan 2001 15:43:57 +0000 >> > + header} >> > + body{ >> > + part{ ID: 1, Content-type: text/plain >> > +Body >> > + part} >> > + body} >> > + message} >> > + message{ id:notmuch-sha1-ca55943aff7a72baf2ab21fa74fab3d632401334 depth:0 match:1 excluded:0 filename:/XXX/mail/msg-1 >> > + header{ >> > +Notmuch Test Suite (1970-01-01) (inbox unread) >> > +Subject: (null) >> > +From: Notmuch Test Suite >> > +Date: Thu, 01 Jan 1970 00:00:00 +0000 >> > + header} >> > + body{ >> > + part{ ID: 1, Content-type: text/plain >> > +Body >> > + part} >> > + body} >> > + message}" >> > + >> > +test_begin_subtest "Show: json" >> > +test_subtest_known_broken >> > +output=$(notmuch show --format=json '*' | notmuch_json_show_sanitize) >> > +test_expect_equal_json "$output" ' >> > +[ >> > + [ >> > + [ >> > + { >> > + "body": [ >> > + { >> > + "content": "Body\n", >> > + "content-type": "text/plain", >> > + "id": 1 >> > + } >> > + ], >> > + "date_relative": "2001-01-05", >> > + "excluded": false, >> > + "filename": "YYYYY", >> > + "headers": { >> > + "Date": "Fri, 05 Jan 2001 15:43:57 +0000", >> > + "From": "", >> > + "Subject": "", >> > + "To": "Notmuch Test Suite " >> > + }, >> > + "id": "XXXXX", >> > + "match": true, >> > + "tags": [ >> > + "inbox", >> > + "unread" >> > + ], >> > + "timestamp": 978709437 >> > + }, >> > + [] >> > + ] >> > + ], >> > + [ >> > + [ >> > + { >> > + "body": [ >> > + { >> > + "content": "Body\n", >> > + "content-type": "text/plain", >> > + "id": 1 >> > + } >> > + ], >> > + "date_relative": "1970-01-01", >> > + "excluded": false, >> > + "filename": "YYYYY", >> > + "headers": { >> > + "Date": "Thu, 01 Jan 1970 00:00:00 +0000", >> > + "From": "Notmuch Test Suite ", >> > + "Subject": "" >> > + }, >> > + "id": "XXXXX", >> > + "match": true, >> > + "tags": [ >> > + "inbox", >> > + "unread" >> > + ], >> > + "timestamp": 0 >> > + }, >> > + [] >> > + ] >> > + ] >> > +]' >> > + >> > + >> > +test_done >> > diff --git a/test/notmuch-test b/test/notmuch-test >> > index ea39dfc..cc732c3 100755 >> > --- a/test/notmuch-test >> > +++ b/test/notmuch-test >> > @@ -59,6 +59,7 @@ TESTS=" >> > emacs-address-cleaning >> > emacs-hello >> > emacs-show >> > + missing-headers >> > " >> > TESTS=${NOTMUCH_TESTS:=$TESTS} >> > >> > >> > _______________________________________________ >> > notmuch mailing list >> > notmuch@notmuchmail.org >> > http://notmuchmail.org/mailman/listinfo/notmuch > _______________________________________________ > notmuch mailing list > notmuch@notmuchmail.org > http://notmuchmail.org/mailman/listinfo/notmuch