From 1ce264245c9dc1bdb6ea17ebf26f38eca284ad20 Mon Sep 17 00:00:00 2001 From: Mark Walters Date: Sat, 1 Nov 2014 00:22:07 +0000 Subject: [PATCH] Re: [PATCH v6 0/7] notmuch search --output=sender/recipients --- 9b/ba3060fa94e27fd008c9b8a76379146490539a | 529 ++++++++++++++++++++++ 1 file changed, 529 insertions(+) create mode 100644 9b/ba3060fa94e27fd008c9b8a76379146490539a diff --git a/9b/ba3060fa94e27fd008c9b8a76379146490539a b/9b/ba3060fa94e27fd008c9b8a76379146490539a new file mode 100644 index 000000000..425d2129f --- /dev/null +++ b/9b/ba3060fa94e27fd008c9b8a76379146490539a @@ -0,0 +1,529 @@ +Return-Path: +X-Original-To: notmuch@notmuchmail.org +Delivered-To: notmuch@notmuchmail.org +Received: from localhost (localhost [127.0.0.1]) + by olra.theworths.org (Postfix) with ESMTP id 17150431FBC + for ; Fri, 31 Oct 2014 17:22:45 -0700 (PDT) +X-Virus-Scanned: Debian amavisd-new at olra.theworths.org +X-Spam-Flag: NO +X-Spam-Score: -1.098 +X-Spam-Level: +X-Spam-Status: No, score=-1.098 tagged_above=-999 required=5 + tests=[DKIM_ADSP_CUSTOM_MED=0.001, FREEMAIL_FROM=0.001, + NML_ADSP_CUSTOM_MED=1.2, RCVD_IN_DNSWL_MED=-2.3] autolearn=disabled +Received: from olra.theworths.org ([127.0.0.1]) + by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) + with ESMTP id KilGQ2SlyKqr for ; + Fri, 31 Oct 2014 17:22:37 -0700 (PDT) +Received: from mail2.qmul.ac.uk (mail2.qmul.ac.uk [138.37.6.6]) + (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) + (No client certificate requested) + by olra.theworths.org (Postfix) with ESMTPS id 1F04B431FAF + for ; Fri, 31 Oct 2014 17:22:37 -0700 (PDT) +Received: from smtp.qmul.ac.uk ([138.37.6.40]) + by mail2.qmul.ac.uk with esmtp (Exim 4.71) + (envelope-from ) + id 1XkMSJ-0000jQ-3Z; Sat, 01 Nov 2014 00:22:35 +0000 +Received: from 5751dfa2.skybroadband.com ([87.81.223.162] helo=localhost) + by smtp.qmul.ac.uk with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.71) + (envelope-from ) + id 1XkMSH-0006JP-Oy; Sat, 01 Nov 2014 00:22:10 +0000 +From: Mark Walters +To: Michal Sojka , notmuch@notmuchmail.org +Subject: Re: [PATCH v6 0/7] notmuch search --output=sender/recipients +In-Reply-To: <1414792441-29555-1-git-send-email-sojkam1@fel.cvut.cz> +References: <1414792441-29555-1-git-send-email-sojkam1@fel.cvut.cz> +User-Agent: Notmuch/0.18.1+86~gef5e66a (http://notmuchmail.org) Emacs/23.4.1 + (x86_64-pc-linux-gnu) +Date: Sat, 01 Nov 2014 00:22:07 +0000 +Message-ID: <87tx2ju5w0.fsf@qmul.ac.uk> +MIME-Version: 1.0 +Content-Type: text/plain; charset=utf-8 +Content-Transfer-Encoding: quoted-printable +X-Sender-Host-Address: 87.81.223.162 +X-QM-Geographic: According to ripencc, + this message was delivered by a machine in Britain (UK) (GB). +X-QM-SPAM-Info: Sender has good ham record. :) +X-QM-Body-MD5: ccef3859bca877dfaa292ff380dd453b (of first 20000 bytes) +X-SpamAssassin-Score: -0.1 +X-SpamAssassin-SpamBar: / +X-SpamAssassin-Report: The QM spam filters have analysed this message to + determine if it is + spam. We require at least 5.0 points to mark a message as spam. + This message scored -0.1 points. + Summary of the scoring: + * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail + provider * (markwalters1009[at]gmail.com) + * -0.1 AWL AWL: From: address is in the auto white-list +X-QM-Scan-Virus: ClamAV says the message is clean +X-BeenThere: notmuch@notmuchmail.org +X-Mailman-Version: 2.1.13 +Precedence: list +List-Id: "Use and development of the notmuch mail system." + +List-Unsubscribe: , + +List-Archive: +List-Post: +List-Help: +List-Subscribe: , + +X-List-Received-Date: Sat, 01 Nov 2014 00:22:45 -0000 + + +On Fri, 31 Oct 2014, Michal Sojka wrote: +> Hi all, +> +> this is v6 of the search --output=3Daddress series. It obsoletes v5 +> (id:1414713573-21461-1-git-send-email-sojkam1@fel.cvut.cz). + +I have tested patches 1-5 and they LGTM. +1 from me. + +Best wishes + +Mark + +> Changes from v5 (full diff below): +> +> - Added quoting of name parts if that is necessary (pointed out by +> Mark Walters). Structured formats contain both full address +> (possibly with quoted name) and unquoted individual fields. +> - Fixed bug in --output=3Dcount --filter-by=3D*fold (reported by Jesse +> Rosenthal). New test was added for this case. Fixing the bug also +> resulted in simpler code :) +> - Added missing unreferencing of InternetAddressList. +> +> Changes from v4: +> +> - patch changed to commit in commit messages +> - opt->format changed to format +> - Added comments to process_* functions +> - duplicite changed to duplicate +> - check_duplicate changed to is_duplicate +> - Deduplication was split into two commits: basic deduplication +> without a command line option and configurable deduplication with +> --fiter-by. +> +> Changes from v3: +> +> - `o' renamed to `opt'. +> - Conversion of --output from keyword to keyword-flags is now a +> separate patch. +> - Structured output formats print name and address separately. +> - Added test for --format=3Djson. +> - Changed --filter-by default to nameaddr. In v2, the default was +> addrfold, in v3 the default was no filtering at all. I believe that +> Mark's suggestion to make nameaddr the default is good trade off. +> - Added new --output=3Dcount +> - Minor style fixes +> - Few typos fixed +> - There is no way to output unfiltered (duplicite) addresses. +> Hopefully, the introduction of --output=3Dcount is sufficient +> replacement for this "feature". +> +> Cheers, +> -Michal +> +> +> Jani Nikula (1): +> cli: Add support for parsing keyword-flag arguments +> +> Michal Sojka (6): +> cli: search: Refactor passing of command line options +> cli: search: Convert --output to keyword-flag argument +> cli: search: Add --output=3D{sender,recipients} +> cli: search: Do not output duplicate addresses +> cli: search: Add --output=3Dcount +> cli: search: Add --filter-by option to configure address filtering +> +> command-line-arguments.c | 6 +- +> command-line-arguments.h | 1 + +> completion/notmuch-completion.bash | 8 +- +> completion/notmuch-completion.zsh | 4 +- +> doc/man1/notmuch-search.rst | 66 ++++++- +> notmuch-search.c | 390 +++++++++++++++++++++++++++++--= +------ +> test/T090-search-output.sh | 87 +++++++++ +> test/T095-search-filter-by.sh | 73 +++++++ +> test/T410-argument-parsing.sh | 3 +- +> test/arg-test.c | 9 + +> 10 files changed, 565 insertions(+), 82 deletions(-) +> create mode 100755 test/T095-search-filter-by.sh +> +> --=20 +> 2.1.1 +> +> diff --git a/notmuch-search.c b/notmuch-search.c +> index 8bc80d3..a350f06 100644 +> --- a/notmuch-search.c +> +++ b/notmuch-search.c +> @@ -246,33 +246,35 @@ is_duplicate (const search_options_t *opt, GHashTab= +le *addrs, const char *name, +> { +> notmuch_bool_t duplicate; +> char *key; +> + gchar *addrfold =3D NULL; +> mailbox_t *mailbox; +>=20=20 +> if (opt->filter_by =3D=3D FILTER_BY_ADDRFOLD || +> - opt->filter_by =3D=3D FILTER_BY_NAMEADDRFOLD) { +> - gchar *folded =3D g_utf8_casefold (addr, -1); +> - addr =3D talloc_strdup (opt->format, folded); +> - g_free (folded); +> - } +> + opt->filter_by =3D=3D FILTER_BY_NAMEADDRFOLD) +> + addrfold =3D g_utf8_casefold (addr, -1); +> + +> switch (opt->filter_by) { +> case FILTER_BY_NAMEADDR: +> - case FILTER_BY_NAMEADDRFOLD: +> key =3D talloc_asprintf (opt->format, "%s <%s>", name, addr); +> break; +> + case FILTER_BY_NAMEADDRFOLD: +> + key =3D talloc_asprintf (opt->format, "%s <%s>", name, addrfold); +> + break; +> case FILTER_BY_NAME: +> key =3D talloc_strdup (opt->format, name); /* !name results in !key */ +> break; +> case FILTER_BY_ADDR: +> - case FILTER_BY_ADDRFOLD: +> key =3D talloc_strdup (opt->format, addr); +> break; +> + case FILTER_BY_ADDRFOLD: +> + key =3D talloc_strdup (opt->format, addrfold); +> + break; +> default: +> INTERNAL_ERROR("invalid --filter-by flags"); +> } +>=20=20 +> - if (opt->filter_by =3D=3D FILTER_BY_ADDRFOLD || +> - opt->filter_by =3D=3D FILTER_BY_NAMEADDRFOLD) +> - talloc_free ((char*)addr); +> + if (addrfold) +> + g_free (addrfold); +>=20=20 +> if (! key) +> return FALSE; +> @@ -300,33 +302,28 @@ print_mailbox (const search_options_t *opt, const m= +ailbox_t *mailbox) +> const char *addr =3D mailbox->addr; +> int count =3D mailbox->count; +> sprinter_t *format =3D opt->format; +> + InternetAddress *ia =3D internet_address_mailbox_new (name, addr); +> + char *name_addr; +>=20=20 +> - if (format->is_text_printer) { +> - char *mailbox_str; +> + /* name_addr has the name part quoted if necessary. Compare +> + * 'John Doe ' vs. '"Doe, John" ' */ +> + name_addr =3D internet_address_to_string (ia, FALSE); +>=20=20 +> - if (name && *name) +> - mailbox_str =3D talloc_asprintf (format, "%s <%s>", name, addr); +> - else +> - mailbox_str =3D talloc_strdup (format, addr); +> - +> - if (! mailbox_str) { +> - fprintf (stderr, "Error: out of memory\n"); +> - return; +> - } +> + if (format->is_text_printer) { +> if (count > 0) { +> format->integer (format, count); +> format->string (format, "\t"); +> } +> - format->string (format, mailbox_str); +> + format->string (format, name_addr); +> format->separator (format); +> - +> - talloc_free (mailbox_str); +> } else { +> format->begin_map (format); +> format->map_key (format, "name"); +> format->string (format, name); +> format->map_key (format, "address"); +> format->string (format, addr); +> + format->map_key (format, "name-addr"); +> + format->string (format, name_addr); +> if (count > 0) { +> format->map_key (format, "count"); +> format->integer (format, count); +> @@ -334,6 +331,9 @@ print_mailbox (const search_options_t *opt, const mai= +lbox_t *mailbox) +> format->end (format); +> format->separator (format); +> } +> + +> + g_object_unref (ia); +> + g_free (name_addr); +> } +>=20=20 +> /* Print or prepare for printing addresses from InternetAddressList. */ +> @@ -389,6 +389,8 @@ process_address_header (const search_options_t *opt, = +GHashTable *addrs, const ch +> return; +>=20=20 +> process_address_list (opt, addrs, list); +> + +> + g_object_unref (list); +> } +>=20=20 +> static void +> diff --git a/test/T090-search-output.sh b/test/T090-search-output.sh +> index 5a9bbc9..82380ac 100755 +> --- a/test/T090-search-output.sh +> +++ b/test/T090-search-output.sh +> @@ -413,73 +413,23 @@ test_expect_equal_file OUTPUT EXPECTED +> test_begin_subtest "--output=3Dsender --format=3Djson" +> notmuch search --output=3Dsender --format=3Djson '*' >OUTPUT +> cat <EXPECTED +> -[{"name": "Fran=C3=A7ois Boulogne", "address": "boulogne.f@gmail.com"}, +> -{"name": "Olivier Berger", "address": "olivier.berger@it-sudparis.eu"}, +> -{"name": "Chris Wilson", "address": "chris@chris-wilson.co.uk"}, +> -{"name": "Carl Worth", "address": "cworth@cworth.org"}, +> -{"name": "Alexander Botero-Lowry", "address": "alex.boterolowry@gmail.co= +m"}, +> -{"name": "Keith Packard", "address": "keithp@keithp.com"}, +> -{"name": "Jjgod Jiang", "address": "gzjjgod@gmail.com"}, +> -{"name": "Rolland Santimano", "address": "rollandsantimano@yahoo.com"}, +> -{"name": "Jan Janak", "address": "jan@ryngle.com"}, +> -{"name": "Stewart Smith", "address": "stewart@flamingspork.com"}, +> -{"name": "Lars Kellogg-Stedman", "address": "lars@seas.harvard.edu"}, +> -{"name": "Alex Botero-Lowry", "address": "alex.boterolowry@gmail.com"}, +> -{"name": "Ingmar Vanhassel", "address": "ingmar@exherbo.org"}, +> -{"name": "Aron Griffis", "address": "agriffis@n01se.net"}, +> -{"name": "Adrian Perez de Castro", "address": "aperez@igalia.com"}, +> -{"name": "Israel Herraiz", "address": "isra@herraiz.org"}, +> -{"name": "Mikhail Gusarov", "address": "dottedmag@dottedmag.net"}] +> -EOF +> -test_expect_equal_file OUTPUT EXPECTED +> - +> -test_begin_subtest "--output=3Dsender --output=3Dcount" +> -notmuch search --output=3Dsender --output=3Dcount '*' | sort -n >OUTPUT +> -cat <EXPECTED +> -1 Adrian Perez de Castro +> -1 Aron Griffis +> -1 Chris Wilson +> -1 Fran=C3=A7ois Boulogne +> -1 Ingmar Vanhassel +> -1 Israel Herraiz +> -1 Olivier Berger +> -1 Rolland Santimano +> -2 Alex Botero-Lowry +> -2 Jjgod Jiang +> -3 Stewart Smith +> -4 Alexander Botero-Lowry +> -4 Jan Janak +> -5 Lars Kellogg-Stedman +> -5 Mikhail Gusarov +> -7 Keith Packard +> -12 Carl Worth +> -EOF +> -test_expect_equal_file OUTPUT EXPECTED +> - +> -test_begin_subtest "--output=3Dsender --output=3Dcount --format=3Djson" +> -# Since the iteration order of GHashTable is not specified, we +> -# preprocess and sort the results to keep the order stable here. +> -notmuch search --output=3Dsender --output=3Dcount --format=3Djson '*' | \ +> - sed -e 's/^\[//' -e 's/]$//' -e 's/,$//' | \ +> - sort --field-separator=3D":" --key=3D4n --key=3D2 >OUTPUT +> -cat <EXPECTED +> -{"name": "Adrian Perez de Castro", "address": "aperez@igalia.com", "coun= +t": 1} +> -{"name": "Aron Griffis", "address": "agriffis@n01se.net", "count": 1} +> -{"name": "Chris Wilson", "address": "chris@chris-wilson.co.uk", "count":= + 1} +> -{"name": "Fran=C3=A7ois Boulogne", "address": "boulogne.f@gmail.com", "c= +ount": 1} +> -{"name": "Ingmar Vanhassel", "address": "ingmar@exherbo.org", "count": 1} +> -{"name": "Israel Herraiz", "address": "isra@herraiz.org", "count": 1} +> -{"name": "Olivier Berger", "address": "olivier.berger@it-sudparis.eu", "= +count": 1} +> -{"name": "Rolland Santimano", "address": "rollandsantimano@yahoo.com", "= +count": 1} +> -{"name": "Alex Botero-Lowry", "address": "alex.boterolowry@gmail.com", "= +count": 2} +> -{"name": "Jjgod Jiang", "address": "gzjjgod@gmail.com", "count": 2} +> -{"name": "Stewart Smith", "address": "stewart@flamingspork.com", "count"= +: 3} +> -{"name": "Alexander Botero-Lowry", "address": "alex.boterolowry@gmail.co= +m", "count": 4} +> -{"name": "Jan Janak", "address": "jan@ryngle.com", "count": 4} +> -{"name": "Lars Kellogg-Stedman", "address": "lars@seas.harvard.edu", "co= +unt": 5} +> -{"name": "Mikhail Gusarov", "address": "dottedmag@dottedmag.net", "count= +": 5} +> -{"name": "Keith Packard", "address": "keithp@keithp.com", "count": 7} +> -{"name": "Carl Worth", "address": "cworth@cworth.org", "count": 12} +> +[{"name": "Fran=C3=A7ois Boulogne", "address": "boulogne.f@gmail.com", "= +name-addr": "Fran=C3=A7ois Boulogne "}, +> +{"name": "Olivier Berger", "address": "olivier.berger@it-sudparis.eu", "= +name-addr": "Olivier Berger "}, +> +{"name": "Chris Wilson", "address": "chris@chris-wilson.co.uk", "name-ad= +dr": "Chris Wilson "}, +> +{"name": "Carl Worth", "address": "cworth@cworth.org", "name-addr": "Car= +l Worth "}, +> +{"name": "Alexander Botero-Lowry", "address": "alex.boterolowry@gmail.co= +m", "name-addr": "Alexander Botero-Lowry "}, +> +{"name": "Keith Packard", "address": "keithp@keithp.com", "name-addr": "= +Keith Packard "}, +> +{"name": "Jjgod Jiang", "address": "gzjjgod@gmail.com", "name-addr": "Jj= +god Jiang "}, +> +{"name": "Rolland Santimano", "address": "rollandsantimano@yahoo.com", "= +name-addr": "Rolland Santimano "}, +> +{"name": "Jan Janak", "address": "jan@ryngle.com", "name-addr": "Jan Jan= +ak "}, +> +{"name": "Stewart Smith", "address": "stewart@flamingspork.com", "name-a= +ddr": "Stewart Smith "}, +> +{"name": "Lars Kellogg-Stedman", "address": "lars@seas.harvard.edu", "na= +me-addr": "Lars Kellogg-Stedman "}, +> +{"name": "Alex Botero-Lowry", "address": "alex.boterolowry@gmail.com", "= +name-addr": "Alex Botero-Lowry "}, +> +{"name": "Ingmar Vanhassel", "address": "ingmar@exherbo.org", "name-addr= +": "Ingmar Vanhassel "}, +> +{"name": "Aron Griffis", "address": "agriffis@n01se.net", "name-addr": "= +Aron Griffis "}, +> +{"name": "Adrian Perez de Castro", "address": "aperez@igalia.com", "name= +-addr": "Adrian Perez de Castro "}, +> +{"name": "Israel Herraiz", "address": "isra@herraiz.org", "name-addr": "= +Israel Herraiz "}, +> +{"name": "Mikhail Gusarov", "address": "dottedmag@dottedmag.net", "name-= +addr": "Mikhail Gusarov "}] +> EOF +> test_expect_equal_file OUTPUT EXPECTED +>=20=20 +> @@ -487,7 +437,7 @@ test_begin_subtest "--output=3Drecipients" +> notmuch search --output=3Drecipients '*' >OUTPUT +> cat <EXPECTED +> Allan McRae +> -Discussion about the Arch User Repository (AUR) +> +"Discussion about the Arch User Repository (AUR)" +> olivier.berger@it-sudparis.eu +> notmuch@notmuchmail.org +> notmuch +> @@ -501,7 +451,7 @@ notmuch search --output=3Dsender --output=3Drecipient= +s '*' >OUTPUT +> cat <EXPECTED +> Fran=C3=A7ois Boulogne +> Allan McRae +> -Discussion about the Arch User Repository (AUR) +> +"Discussion about the Arch User Repository (AUR)" +> Olivier Berger +> olivier.berger@it-sudparis.eu +> Chris Wilson +> diff --git a/test/T095-search-filter-by.sh b/test/T095-search-filter-by.sh +> index 97d9a9b..15c9f77 100755 +> --- a/test/T095-search-filter-by.sh +> +++ b/test/T095-search-filter-by.sh +> @@ -2,17 +2,17 @@ +> test_description=3D'duplicite address filtering in "notmuch search --out= +put=3Drecipients"' +> . ./test-lib.sh +>=20=20 +> -add_message '[to]=3D"Real Name , Real Name "' +> -add_message '[to]=3D"Nickname "' '[cc]=3D"Real Name "' +> -add_message '[to]=3D"Nickname "' '[bcc]=3D"Real Name "' +> +add_message '[to]=3D"John Doe , John Doe "' +> +add_message '[to]=3D"\"Doe, John\" "' '[cc]=3D"John Doe= + "' +> +add_message '[to]=3D"\"Doe, John\" "' '[bcc]=3D"John Do= +e "' +>=20=20 +> test_begin_subtest "--output=3Drecipients" +> notmuch search --output=3Drecipients "*" >OUTPUT +> cat <EXPECTED +> -Real Name +> -Real Name +> -Nickname +> -Real Name +> +John Doe +> +John Doe +> +"Doe, John" +> +John Doe +> EOF +> test_expect_equal_file OUTPUT EXPECTED +>=20=20 +> @@ -20,44 +20,53 @@ test_begin_subtest "--output=3Drecipients --filter-by= +=3Dnameaddr" +> notmuch search --output=3Drecipients --filter-by=3Dnameaddr "*" >OUTPUT +> # The same as above +> cat <EXPECTED +> -Real Name +> -Real Name +> -Nickname +> -Real Name +> +John Doe +> +John Doe +> +"Doe, John" +> +John Doe +> EOF +> test_expect_equal_file OUTPUT EXPECTED +>=20=20 +> test_begin_subtest "--output=3Drecipients --filter-by=3Dname" +> notmuch search --output=3Drecipients --filter-by=3Dname "*" >OUTPUT +> cat <EXPECTED +> -Real Name +> -Nickname +> +John Doe +> +"Doe, John" +> EOF +> test_expect_equal_file OUTPUT EXPECTED +>=20=20 +> test_begin_subtest "--output=3Drecipients --filter-by=3Daddr" +> notmuch search --output=3Drecipients --filter-by=3Daddr "*" >OUTPUT +> cat <EXPECTED +> -Real Name +> -Real Name +> -Real Name +> +John Doe +> +John Doe +> +John Doe +> EOF +> test_expect_equal_file OUTPUT EXPECTED +>=20=20 +> test_begin_subtest "--output=3Drecipients --filter-by=3Daddrfold" +> notmuch search --output=3Drecipients --filter-by=3Daddrfold "*" >OUTPUT +> cat <EXPECTED +> -Real Name +> -Real Name +> +John Doe +> +John Doe +> EOF +> test_expect_equal_file OUTPUT EXPECTED +>=20=20 +> test_begin_subtest "--output=3Drecipients --filter-by=3Dnameaddrfold" +> notmuch search --output=3Drecipients --filter-by=3Dnameaddrfold "*" >OUT= +PUT +> cat <EXPECTED +> -Real Name +> -Real Name +> -Nickname +> +John Doe +> +John Doe +> +"Doe, John" +> +EOF +> +test_expect_equal_file OUTPUT EXPECTED +> + +> +test_begin_subtest "--output=3Drecipients --filter-by=3Dnameaddrfold --o= +utput=3Dcount" +> +notmuch search --output=3Drecipients --filter-by=3Dnameaddrfold --output= +=3Dcount "*" | sort -n >OUTPUT +> +cat <EXPECTED +> +1 John Doe +> +2 "Doe, John" +> +3 John Doe +> EOF +> test_expect_equal_file OUTPUT EXPECTED +>=20=20 +> _______________________________________________ +> notmuch mailing list +> notmuch@notmuchmail.org +> http://notmuchmail.org/mailman/listinfo/notmuch -- 2.26.2