From: Mark Walters <markwalters1009@gmail.com>
To: Jameson Graef Rollins <jrollins@finestructure.net>,
 notmuch@notmuchmail.org,	Austin Clements <amdragon@mit.edu>
Subject: Re: [RFC PATCH v5 00/11] Add NOTMUCH_MESSAGE_FLAG_EXCLUDED flag
In-Reply-To: <874nurrbdb.fsf@servo.finestructure.net>
References: <1329296619-7463-1-git-send-email-markwalters1009@gmail.com>
	<8739acrnu7.fsf@servo.finestructure.net>
	<8739aber9o.fsf@qmul.ac.uk>
	<874nurrbdb.fsf@servo.finestructure.net>
User-Agent: Notmuch/0.11.1+206~g3b67774 (http://notmuchmail.org) Emacs/23.2.1
	(i486-pc-linux-gnu)
Date: Thu, 16 Feb 2012 00:30:35 +0000
Message-ID: <87aa4jtyac.fsf@qmul.ac.uk>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Precedence: list

On Wed, 15 Feb 2012 14:16:16 -0800, Jameson Graef Rollins <jrollins@finestructure.net> wrote:
> On Wed, 15 Feb 2012 21:11:15 +0000, Mark Walters <markwalters1009@gmail.com> wrote:
> > I think the difficulty is that there are lots of annoying corner cases,
> > but if there is a  simpler solution that would be great!
> 
> I think there is!
> 
> > 1) What should notmuch show id:deleted-message-id do? 
> > 
> > It could return the thread containing the deleted message. If it does
> > return a thread what subject does it assign it?  Possibly it could
> > return no messages and the caller would have to call it again with
> > --no-exclude.
> 
> "notmuch show id:<excluded-id>" should always return the message
> matching id:<excluded-id> with match=true.  In fact, any search that
> references a specific id: should always process the message as if there
> were no excludes at all.
> 
> Excluded messages are not directly accessible at the moment, which is
> definitely a bug.  Adding the --no-excludes option will help, but I
> still think we should just implement the behavior I outline above.
> 
> > 2) Should notmuch search return threads which match but only in excluded
> > messages? 
> > 
> > If yes then does it sort it based on match or match-not-excluded? If the
> > latter what happens to threads with no match-not-excluded messages? If
> > not then does searching for id:deleted-message return no results? The
> > caller could try with --no-exclude, but then the caller would end up
> > returning the deleted message for search id:deleted-message but not for
> > search id:deleted-message or id:some-other-not-deleted-message.
> 
> See the point above.  If one of the search terms is an id: then that
> message should be returned matched, as if there were no excludes.
> 
> I think this is the right solution, for both search and show:
> 
> - excluded messages are just match=false
> 
> - searches that reference a specific id: are match=true no matter what
>   their exclude status
> 
> - searches that reference an excluded tag are match=true
> 
> As far as I can see this should "just work", without any existing
> changes to consumers.  Anyone see any issues I'm missing?

So is the following what you are suggesting:

always return all messages/threads that match the query whether or not
the match is excluded, but only mark the message as a match if it
matches and is not excluded (with some exceptions: e.g., for id: queries
and tag:deleted queries).

A different way of looking at this is that we revert to the old
pre-exclude behaviour and then implement excludes as a "turn off the
match flag" rather than the current "omit the results".

That sounds plausible. 

But I am not sure how much simpler it is. Of my patch set patches 1-3
are essentially separate (the --no-exclude bits). 

I think you would need patches 4 and 5 or something similar since we
both need to mark the excluded messages ("excluded" in my case, "not
matched" in your case).

Patch 6 could be simplified: since in your case the exclude status only
matters on matching messages we have already computed it (although the
exclude information would need to passed to create_thread)

Patch 7: loses the exclude flag bits (5 lines or so) but the bulk is the
--no-exclude for show.

Patch 8 man-page stays

Patch 9 (updating tests) might not be needed since the output is
changing less.

Patch 10 the "omit" excluded results totally option. What would you do
for notmuch search --output=messages? either it outputs all matching
messages or only those that match and are not excluded. The code can of
course check, but things like --output=tags are more annoying as that is
back into the library code. In any event I think something here is
needed, and in fact any solution here would apply to either version.

Patch 11: emacs would be a bit simpler in your case.

However, you would need to special case the id: stuff (which could
probably be done in Austin's code that examines the parsed query for
tags). 


One possibility would be for my patch to set the match status based on
match flag and not exclude flag, and separately return the whole
flag-set. This might give nicer behaviour for current callers whilst
giving the full information to new callers.

(This email was mostly written when Austin sent his)

Best wishes

Mark