Received-SPF: pass (google.com: domain of markwalters1009@gmail.com designates
	10.180.24.202 as permitted sender) client-ip=10.180.24.202; 
From: Mark Walters <markwalters1009@gmail.com>
To: notmuch@notmuchmail.org,
	Austin Clements <amdragon@MIT.EDU>
Subject: [Patch v6 00/13] Add NOTMUCH_MESSAGE_FLAG_EXCLUDED flag
Date: Sat, 25 Feb 2012 08:06:31 +0000
Message-Id: <1330157204-26094-1-git-send-email-markwalters1009@gmail.com>
Precedence: list


Here is the latest version of the series. It fixes all of Austin's
review comments. I don't think there are any significant outstanding
issues.

Having asked whether excluded messages should still be marked excluded
when the tag is explicitly mentioned in the query I realised that they
must not be marked. If they are then the front end cannot distinguish
messages that are excluded because they match the tag the user has
said they do not care about (eg tag:deleted), or excluded because they
match some other excluded tag (eg tag:spam).

I have also updated the test for search since the matching only in an
excluded message thread is now returned but the "match count" is zero
so the frontend cannot ignore it if it wishes.

Finally, I updated notmuch.el so that it throws away threads with zero
matches.  As mentioned in the commit comment there are better
possibilities but this is a minimal change (it is a 1 line patch
except for indentation).

The total changes from v5 are:
    
    Three changes of search_exclude_tags to search.exclude_tags in the
    man pages. (one in each of count/show and search).

    Changed the talloc context of the excluded messages to message
    rather than query.

    Corrected an indentation error (8 spaces instead of a tab)
    
    Moved a test patch earlier in the series (next to its
    corresponding change); no change overall.

    Fixed the search test.
     
    Remove exclude tags that occur in the query from the list of
    exclude tags so that the messages do not get marked excluded.

    Make notmuch.el drop results with no matching non-excluded
    messages.

The diff from v5 is at the end of this cover letter.

Best wishes 

Mark


Mark Walters (13):
  cli: add --no-exclude option to count and search.
  cli: Add --no-exclude to the man pages for search and count
  test: add tests for new cli --no-exclude option
  lib: Rearrange the exclude code in query.cc
  lib: Make notmuch_query_search_messages set the exclude flag
  lib: Add the exclude flag to notmuch_query_search_threads
  test: update search test to reflect exclude flag
  cli: Make notmuch-show respect excludes.
  test: update tests to reflect the exclude flag
  man: update manpage for notmuch-show --no-exclude option
  cli: omit excluded messages in results where appropriate.
  emacs: show: recognize the exclude flag.
  emacs: notmuch.el ignore excluded matches

 emacs/notmuch-show.el     |   19 ++++++++++-
 emacs/notmuch.el          |   22 +++++++------
 lib/notmuch-private.h     |    8 +++-
 lib/notmuch.h             |   16 +++++++--
 lib/query.cc              |   77 +++++++++++++++++++++++++++++++++++++++------
 lib/thread.cc             |   18 +++++++++--
 man/man1/notmuch-count.1  |    7 ++++
 man/man1/notmuch-search.1 |    7 ++++
 man/man1/notmuch-show.1   |    7 ++++
 notmuch-count.c           |   19 +++++++---
 notmuch-search.c          |   26 ++++++++++++---
 notmuch-show.c            |   31 ++++++++++++++++--
 test/count                |   21 ++++++++++++
 test/crypto               |    9 +++++-
 test/encoding             |    2 +-
 test/json                 |    6 ++--
 test/maildir-sync         |    1 +
 test/multipart            |    4 +-
 test/search               |    8 ++++-
 test/thread-naming        |   16 +++++-----
 20 files changed, 263 insertions(+), 61 deletions(-)

diff --git a/emacs/notmuch.el b/emacs/notmuch.el
index 5b4f1c5..8b209cd 100644
--- a/emacs/notmuch.el
+++ b/emacs/notmuch.el
@@ -865,16 +865,18 @@ non-authors is found, assume that all of the authors match."
 		      (goto-char (point-max))
 		      (if (/= (match-beginning 1) line)
 			  (insert (concat "Error: Unexpected output from notmuch search:\n" (substring string line (match-beginning 1)) "\n")))
-		      (let ((beg (point)))
-			(notmuch-search-show-result date count authors
-						    (notmuch-prettify-subject subject) tags)
-			(notmuch-search-color-line beg (point) tag-list)
-			(put-text-property beg (point) 'notmuch-search-thread-id thread-id)
-			(put-text-property beg (point) 'notmuch-search-authors authors)
-			(put-text-property beg (point) 'notmuch-search-subject subject)
-			(when (string= thread-id notmuch-search-target-thread)
-			  (set 'found-target beg)
-			  (set 'notmuch-search-target-thread "found")))
+		      ;; We currently just throw away excluded matches.
+		      (unless (eq (aref count 1) ?0)
+			(let ((beg (point)))
+			  (notmuch-search-show-result date count authors
+						      (notmuch-prettify-subject subject) tags)
+			  (notmuch-search-color-line beg (point) tag-list)
+			  (put-text-property beg (point) 'notmuch-search-thread-id thread-id)
+			  (put-text-property beg (point) 'notmuch-search-authors authors)
+			  (put-text-property beg (point) 'notmuch-search-subject subject)
+			  (when (string= thread-id notmuch-search-target-thread)
+			    (set 'found-target beg)
+			    (set 'notmuch-search-target-thread "found"))))
 		      (set 'line (match-end 0)))
 		  (set 'more nil)
 		  (while (and (< line (length string)) (= (elt string line) ?\n))
diff --git a/lib/query.cc b/lib/query.cc
index e1c3977..ab18fbc 100644
--- a/lib/query.cc
+++ b/lib/query.cc
@@ -139,8 +139,9 @@ _notmuch_messages_destructor (notmuch_mset_messages_t *messages)
 
 /* Return a query that matches messages with the excluded tags
  * registered with query.  Any tags that explicitly appear in xquery
- * will not be excluded. The caller of this function has to combine
- * the returned query appropriately.*/
+ * will not be excluded, and will be removed from the list of exclude
+ * tags.  The caller of this function has to combine the returned
+ * query appropriately.*/
 static Xapian::Query
 _notmuch_exclude_tags (notmuch_query_t *query, Xapian::Query xquery)
 {
@@ -157,6 +158,8 @@ _notmuch_exclude_tags (notmuch_query_t *query, Xapian::Query xquery)
 	if (it == end)
 	    exclude_query = Xapian::Query (Xapian::Query::OP_OR,
 				    exclude_query, Xapian::Query (term->string));
+	else
+	    term->string = talloc_strdup (query, "");
     }
     return exclude_query;
 }
@@ -228,7 +231,7 @@ notmuch_query_search_messages (notmuch_query_t *query)
 		    unsigned int doc_id = *iterator;
 		    g_array_append_val (excluded_doc_ids, doc_id);
 		}
-		messages->base.excluded_doc_ids = talloc (query, _notmuch_doc_id_set);
+		messages->base.excluded_doc_ids = talloc (messages, _notmuch_doc_id_set);
 		_notmuch_doc_id_set_init (query, messages->base.excluded_doc_ids,
 					  excluded_doc_ids);
 		g_array_unref (excluded_doc_ids);
diff --git a/man/man1/notmuch-count.1 b/man/man1/notmuch-count.1
index 413b405..97972a7 100644
--- a/man/man1/notmuch-count.1
+++ b/man/man1/notmuch-count.1
@@ -43,7 +43,7 @@ Output the number of matching threads.
 .TP 4
 .BR \-\-no\-exclude
 
-Do not exclude the messages matching search_exclude_tags in the config file.
+Do not exclude the messages matching search.exclude_tags in the config file.
 .RE
 .RE
 .RE
diff --git a/man/man1/notmuch-search.1 b/man/man1/notmuch-search.1
index bc54b4d..f2e5a38 100644
--- a/man/man1/notmuch-search.1
+++ b/man/man1/notmuch-search.1
@@ -116,7 +116,7 @@ Limit the number of displayed results to N.
 .TP 4
 .BR \-\-no\-exclude
 
-Do not exclude the messages matching search_exclude_tags in the config file.
+Do not exclude the messages matching search.exclude_tags in the config file.
 .RE
 
 .SH SEE ALSO
diff --git a/man/man1/notmuch-show.1 b/man/man1/notmuch-show.1
index f743a53..8383a05 100644
--- a/man/man1/notmuch-show.1
+++ b/man/man1/notmuch-show.1
@@ -132,7 +132,7 @@ content.
 .TP 4
 .B \-\-no-exclude
 
-Do not exclude the messages matching search_exclude_tags in the config file.
+Do not exclude the messages matching search.exclude_tags in the config file.
 .RE
 
 A common use of
diff --git a/notmuch-search.c b/notmuch-search.c
index d2b2488..61c65e5 100644
--- a/notmuch-search.c
+++ b/notmuch-search.c
@@ -454,7 +454,7 @@ notmuch_search_command (void *ctx, int argc, char *argv[])
 				  { "files", OUTPUT_FILES },
 				  { "tags", OUTPUT_TAGS },
 				  { 0, 0 } } },
-        { NOTMUCH_OPT_BOOLEAN, &no_exclude, "no-exclude", 'd', 0 },
+	{ NOTMUCH_OPT_BOOLEAN, &no_exclude, "no-exclude", 'd', 0 },
 	{ NOTMUCH_OPT_INT, &offset, "offset", 'O', 0 },
 	{ NOTMUCH_OPT_INT, &limit, "limit", 'L', 0  },
 	{ 0, 0, 0, 0, 0 }
diff --git a/test/search b/test/search
index 3da5d17..081f60c 100755
--- a/test/search
+++ b/test/search
@@ -136,7 +136,8 @@ generate_message '[subject]="Deleted"'
 notmuch new > /dev/null
 notmuch tag +deleted id:$gen_msg_id
 output=$(notmuch search subject:deleted | notmuch_search_sanitize)
-test_expect_equal "$output" "thread:XXX   2001-01-05 [1/1] Notmuch Test Suite; Not deleted (inbox unread)"
+test_expect_equal "$output" "thread:XXX   2001-01-05 [1/1] Notmuch Test Suite; Not deleted (inbox unread)
+thread:XXX   2001-01-05 [0/1] Notmuch Test Suite; Deleted (deleted inbox unread)"
 
 test_begin_subtest "Exclude \"deleted\" messages from search, overridden"
 output=$(notmuch search subject:deleted and tag:deleted | notmuch_search_sanitize)