Date: Thu, 30 Jan 2014 17:02:34 -0500
From: Austin Clements <amdragon@MIT.EDU>
To: Jani Nikula <jani@nikula.org>
Subject: Re: [PATCH 0/5] lib: make folder: prefix literal
Message-ID: <20140130220234.GI4375@mit.edu>
References: <cover.1389304779.git.jani@nikula.org>
	<87y525m649.fsf@awakening.csail.mit.edu>
	<87r47wfltb.fsf@nikula.org> <87iot8f4vg.fsf@nikula.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <87iot8f4vg.fsf@nikula.org>
User-Agent: Mutt/1.5.21 (2010-09-15)
Cc: notmuch@notmuchmail.org
Precedence: list

Quoth Jani Nikula on Jan 25 at  5:38 pm:
> On Sat, 25 Jan 2014, Jani Nikula <jani@nikula.org> wrote:
> > Perhaps we need to have two prefixes, one of which is the literal
> > filesystem folder and another which hides the implementation details,
> > like I mentioned in my mail to Peter [1]. But consider this: my proposed
> > implementation does cover *all* use cases.
> 
> Here's a thought. With boolean prefix folder:, we can devise a scheme
> where the folder: query defines what is to be matched.
> 
> For example:
> 
> folder:foo	match files in foo, foo/new, and foo/cur.
> folder:foo/	match all files in all subdirectories under foo (this
> 		would handle Tomi's use case), including foo/new and foo/cur.
> folder:foo/.	match in foo only, and specifically not in foo/cur or foo/new.
> folder:foo/new  match in foo/new, and specifically not in foo/cur (this
> 		allows distinguishing between messages in cur and new).
> folder:/	match everything.
> folder:/.	match in top level maildir only.
> folder:""	match in top level maildir, including cur/new.
> 
> This requires indexing all the path components with suitable
> suffixes. For example, a file "foo/new/baz" would get terms "/", "foo",
> "foo/", "foo/new", and "foo/new/.". A file foo/bar would get terms "/",
> "foo", "foo/", and "foo/.".
> 
> It's obviously a concern this increases the database size; not sure how
> it would compare with the current stemmed probabilistic prefix.
> 
> Opinions on this? This would really cover all use cases, and address
> Austin's interface and backward compatibility concerns.

I like this idea in general, though I agree with others that the
specific syntax seems a little wanting.  The concept of adding several
boolean terms seems powerful, and I would be surprised if the extra
terms had any substantive effect on database size.

However, it seems like this is overloading one prefix for two
meanings.  And I think that's because people want two similar but
distinct things.  Several of us want a simple, natural Maildir-aware
folder search (the Maildir folder of "a/b/cur/x:2," is "a/b").  Others
want file system search.  It's easy to conflate these because Maildir
represents folders as directory paths, but maybe they need to be
treated as distinct things.

What if we introduce two prefixes, say folder: and path: (maybe dir:?)
to address both use cases, each as naturally as possible?  Both would
be boolean prefixes because of the limitations of probabilistic
prefixes, but we could take advantage of Jani's idea of generating
several boolean terms.

folder: could work the way I suggested (simply the path to the file,
with {cur,new} stripped off).  path: would support file system search
uses.  These seem more varied, but I think fall into exact match and
recursive match.  Since I don't have this use case, I can't have any
strong opinions about syntax, but I'll throw out an idea: many shells
support "**" for recursive path matching and people are already quite
familiar with glob patterns for paths, so why not simply adopt this?
In other words, when adding the path "a/b/cur/x:2," add path: terms
"a/b/cur" and "a/b/**" and "a/**" and "**".

> BR,
> Jani.