Re: [PATCH] lib: Add a new prefix "list" to the search-terms syntax

diff --git a/98/751f4487ae63899aefa12f1229554871b0dcc2 b/98/751f4487ae63899aefa12f1229554871b0dcc2
new file mode 100644 (file)
index 0000000..a823d8e
--- /dev/null
+++ b/98/751f4487ae63899aefa12f1229554871b0dcc2
@@ -0,0 +1,118 @@
+Return-Path: <david@tethera.net>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+       by olra.theworths.org (Postfix) with ESMTP id EB318431FB6\r
+       for <notmuch@notmuchmail.org>; Mon, 29 Apr 2013 18:12:32 -0700 (PDT)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: 0\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=0 tagged_above=-999 required=5 tests=[none]\r
+       autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+       by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+       with ESMTP id 5c9T9YLBPAOG for <notmuch@notmuchmail.org>;\r
+       Mon, 29 Apr 2013 18:12:28 -0700 (PDT)\r
+Received: from tesseract.cs.unb.ca (tesseract.cs.unb.ca [131.202.240.238])\r
+       (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits))\r
+       (No client certificate requested)\r
+       by olra.theworths.org (Postfix) with ESMTPS id 5C71D431FAF\r
+       for <notmuch@notmuchmail.org>; Mon, 29 Apr 2013 18:12:28 -0700 (PDT)\r
+Received: from fctnnbsc30w-156034082078.dhcp-dynamic.fibreop.nb.bellaliant.net\r
+       ([156.34.82.78] helo=zancas.localnet)\r
+       by tesseract.cs.unb.ca with esmtpsa\r
+       (TLS1.2:DHE_RSA_AES_128_CBC_SHA1:128) (Exim 4.80)\r
+       (envelope-from <david@tethera.net>)\r
+       id 1UWz7G-0004a6-VW; Mon, 29 Apr 2013 22:12:25 -0300\r
+Received: from bremner by zancas.localnet with local (Exim 4.80)\r
+       (envelope-from <david@tethera.net>)\r
+       id 1UWz7A-0003yr-SL; Mon, 29 Apr 2013 22:12:16 -0300\r
+From: David Bremner <david@tethera.net>\r
+To: "Alexey I. Froloff" <raorn@raorn.name>, notmuch@notmuchmail.org\r
+Subject: Re: [PATCH] lib: Add a new prefix "list" to the search-terms syntax\r
+In-Reply-To: <1365549369-12776-1-git-send-email-raorn@raorn.name>\r
+References: <20130409083010.GA27675@raorn.name>\r
+       <1365549369-12776-1-git-send-email-raorn@raorn.name>\r
+User-Agent: Notmuch/0.15.2+84~g12d5e4e (http://notmuchmail.org) Emacs/24.2.1\r
+       (x86_64-pc-linux-gnu)\r
+Date: Mon, 29 Apr 2013 22:12:16 -0300\r
+Message-ID: <87vc74vn2n.fsf@zancas.localnet>\r
+MIME-Version: 1.0\r
+Content-Type: text/plain\r
+X-Spam_bar: -\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+       <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Tue, 30 Apr 2013 01:12:33 -0000\r
+\r
+\r
+Hi Alexey, \r
+\r
+Thanks for working on this. I think the boolean prefix version makes\r
+more sense, and it seems to work OK. I have a few comments below\r
+\r
+"Alexey I. Froloff" <raorn@raorn.name> writes:\r
+\r
+> +    begin_list_id = strrchr (list_id_header, '<');\r
+> +    if (!begin_list_id) {\r
+> +    fprintf (stderr, "Warning: Not indexing mailformed List-Id tag.\n");\r
+> +    return;\r
+> +    }\r
+\r
+- I guess this should say "malformed". \r
+\r
+- I got about 1800 lines of such messages when indexing 280k\r
+  messages. That might strike some people as excessive. On the otherhand\r
+  I guess we need to re-think error reporting overall.\r
+\r
+  What do you think about printing filename or message-id here its\r
+  easier to double check that it is not a bug?\r
+\r
+> +    end_list_id = strrchr(begin_list_id, '>');\r
+> +    if (!end_list_id || (end_list_id - begin_list_id < 2)) {\r
+> +    fprintf (stderr, "Warning: Not indexing mailformed List-Id tag.\n");\r
+> +    return;\r
+> +    }\r
+> +\r
+\r
+Same comments here.\r
+\r
+> +    void *local = talloc_new (message);\r
+> +\r
+> +    /* We extract the list id between the angle brackets */\r
+> +    const char *list_id = talloc_strndup (local, begin_list_id + 1,\r
+> +                                      end_list_id - begin_list_id - 1);\r
+> +\r
+      we should handle ENOMEM here, I think.\r
+\r
+> +    /* _notmuch_message_add_term() may return\r
+> +     * NOTMUCH_PRIVATE_STATUS_TERM_TOO_LONG here.  We can't fix it, but\r
+> +     * this is not a reason to exit with error... */\r
+> +    if (_notmuch_message_add_term (message, "list", list_id))\r
+> +    fprintf (stderr, "Warning: Not indexing List-Id: <%s>\n", list_id);\r
+\r
+This should say why the indexing failed.\r
+\r
+Other than that:\r
+\r
+- We need a couple tests for this code; tests/search should give some\r
+  hints how to proceed.\r
+\r
+- We need a patch for NEWS, explaining what people need to do take\r
+  advantage of the new functionality.  I think that adding new prefixes\r
+  to an existing database is OK, but I'd welcome confirmation.\r
+\r
+BTW, my not too scientific tests show no detectable bloat in the\r
+database, at least after running xapian-compact. I'd be curious what\r
+other people report.\r
+\r