From: Jani Nikula <jani@nikula.org>
To: "Alexey I. Froloff" <raorn@raorn.name>, notmuch@notmuchmail.org
Subject: Re: [PATCH] lib: Add a new prefix "list" to the search-terms syntax
In-Reply-To: <1365549369-12776-1-git-send-email-raorn@raorn.name>
References: <20130409083010.GA27675@raorn.name>
	<1365549369-12776-1-git-send-email-raorn@raorn.name>
User-Agent: Notmuch/0.16+97~g6878b0b (http://notmuchmail.org) Emacs/24.3.1
	(x86_64-pc-linux-gnu)
Date: Thu, 17 Oct 2013 17:17:00 +0300
Message-ID: <87bo2ougmb.fsf@nikula.org>
MIME-Version: 1.0
Content-Type: text/plain
Cc: "Alexey I. Froloff" <raorn@raorn.name>
Precedence: list

On Wed, 10 Apr 2013, "Alexey I. Froloff" <raorn@raorn.name> wrote:
> From: "Alexey I. Froloff" <raorn@raorn.name>
>
> Add support for indexing and searching the message's List-Id header.
> This is useful when matching all the messages belonging to a particular
> mailing list.

There's an issue with our duplicate message-id handling that is likely
to cause confusion with List-Id: searches. If you receive several
duplicates of the same message (judged by the message-id), only the
first one of them gets indexed, and the rest are ignored. This means
that for messages you receive both directly and through a list, it will
be arbitrary whether the List-Id: gets indexed or not. Therefore a list:
search might not return all the messages you'd expect.

BR,
Jani.


> Rework of the patch by Pablo Oliveira <pablo@sifflez.org>
>
> Differences from original patch:
>
> The whole list ID indexed as boolean term, not split by words.
> List description is not indexed at all.
>
> Thanks to ojwb and amdragon from irc://irc.freenode.net/notmuch
>
> Signed-off-by: Alexey I. Froloff <raorn@raorn.name>
> ---
>  lib/database.cc                 |  1 +
>  lib/index.cc                    | 45 ++++++++++++++++++++++++++++++++++++++++-
>  man/man7/notmuch-search-terms.7 |  8 ++++++++
>  3 files changed, 53 insertions(+), 1 deletion(-)
>
> diff --git a/lib/database.cc b/lib/database.cc
> index 91d4329..6313913 100644
> --- a/lib/database.cc
> +++ b/lib/database.cc
> @@ -203,6 +203,7 @@ static prefix_t BOOLEAN_PREFIX_INTERNAL[] = {
>  };
>  
>  static prefix_t BOOLEAN_PREFIX_EXTERNAL[] = {
> +    { "list",			"XLIST"},
>      { "thread",			"G" },
>      { "tag",			"K" },
>      { "is",			"K" },
> diff --git a/lib/index.cc b/lib/index.cc
> index a2edd6d..8b97ec3 100644
> --- a/lib/index.cc
> +++ b/lib/index.cc
> @@ -304,6 +304,46 @@ _index_address_list (notmuch_message_t *message,
>      }
>  }
>  
> +static void
> +_index_list_id (notmuch_message_t *message,
> +               const char *list_id_header)
> +{
> +    const char *begin_list_id, *end_list_id;
> +
> +    if (list_id_header == NULL)
> +	return;
> +
> +    /* RFC2919 says that the list-id is found at the end of the header
> +     * and enclosed between angle brackets. If we cannot find a
> +     * matching pair of brackets containing at least one character,
> +     * we ignore the list id header. */
> +    begin_list_id = strrchr (list_id_header, '<');
> +    if (!begin_list_id) {
> +	fprintf (stderr, "Warning: Not indexing mailformed List-Id tag.\n");
> +	return;
> +    }
> +
> +    end_list_id = strrchr(begin_list_id, '>');
> +    if (!end_list_id || (end_list_id - begin_list_id < 2)) {
> +	fprintf (stderr, "Warning: Not indexing mailformed List-Id tag.\n");
> +	return;
> +    }
> +
> +    void *local = talloc_new (message);
> +
> +    /* We extract the list id between the angle brackets */
> +    const char *list_id = talloc_strndup (local, begin_list_id + 1,
> +					  end_list_id - begin_list_id - 1);
> +
> +    /* _notmuch_message_add_term() may return
> +     * NOTMUCH_PRIVATE_STATUS_TERM_TOO_LONG here.  We can't fix it, but
> +     * this is not a reason to exit with error... */
> +    if (_notmuch_message_add_term (message, "list", list_id))
> +	fprintf (stderr, "Warning: Not indexing List-Id: <%s>\n", list_id);
> +
> +    talloc_free (local);
> +}
> +
>  /* Callback to generate terms for each mime part of a message. */
>  static void
>  _index_mime_part (notmuch_message_t *message,
> @@ -432,7 +472,7 @@ _notmuch_message_index_file (notmuch_message_t *message,
>      GMimeMessage *mime_message = NULL;
>      InternetAddressList *addresses;
>      FILE *file = NULL;
> -    const char *from, *subject;
> +    const char *from, *subject, *list_id;
>      notmuch_status_t ret = NOTMUCH_STATUS_SUCCESS;
>      static int initialized = 0;
>      char from_buf[5];
> @@ -500,6 +540,9 @@ mboxes is deprecated and may be removed in the future.\n", filename);
>      subject = g_mime_message_get_subject (mime_message);
>      _notmuch_message_gen_terms (message, "subject", subject);
>  
> +    list_id = g_mime_object_get_header (GMIME_OBJECT (mime_message), "List-Id");
> +    _index_list_id (message, list_id);
> +
>      _index_mime_part (message, g_mime_message_get_mime_part (mime_message));
>  
>    DONE:
> diff --git a/man/man7/notmuch-search-terms.7 b/man/man7/notmuch-search-terms.7
> index eb417ba..9cae107 100644
> --- a/man/man7/notmuch-search-terms.7
> +++ b/man/man7/notmuch-search-terms.7
> @@ -52,6 +52,8 @@ terms to match against specific portions of an email, (where
>  
>  	thread:<thread-id>
>  
> +	list:<list-id>
> +
>  	folder:<directory-path>
>  
>  	date:<since>..<until>
> @@ -100,6 +102,12 @@ thread ID values can be seen in the first column of output from
>  .B "notmuch search"
>  
>  The
> +.BR list: ,
> +is used to match mailing list ID of an email message \- contents of the
> +List\-Id: header without the '<', '>' delimiters or decoded list
> +description.
> +
> +The
>  .B folder:
>  prefix can be used to search for email message files that are
>  contained within particular directories within the mail store. Only
> -- 
> 1.8.1.4
>
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch