From: Tomi Ollila <tomi.ollila@iki.fi>
To: Jani Nikula <jani@nikula.org>, Austin Clements <aclements@csail.mit.edu>,
	notmuch@notmuchmail.org
Subject: Re: [PATCH 0/5] lib: make folder: prefix literal
In-Reply-To: <87r47wfltb.fsf@nikula.org>
References: <cover.1389304779.git.jani@nikula.org>
	<87y525m649.fsf@awakening.csail.mit.edu>
	<87r47wfltb.fsf@nikula.org>
User-Agent: Notmuch/0.17+41~g8e7fabf (http://notmuchmail.org) Emacs/24.3.1
	(x86_64-unknown-linux-gnu)
Date: Sat, 25 Jan 2014 12:47:13 +0200
Message-ID: <m2zjmks5hq.fsf@guru.guru-group.fi>
MIME-Version: 1.0
Content-Type: text/plain
Precedence: list

On Sat, Jan 25 2014, Jani Nikula <jani@nikula.org> wrote:

> On Fri, 24 Jan 2014, Austin Clements <aclements@csail.mit.edu> wrote:
>> On Thu, 09 Jan 2014, Jani Nikula <jani@nikula.org> wrote:
>>> Hi all, this series makes the folder: search prefix literal, or switches
>>> it from a probabilistic prefix to a boolean prefix. With this, you have
>>> to give the path from the maildir root to the folder you want in full,
>>> including the maildir cur/new component, if any. Examples:
>>
>> I strongly disagree with requiring the cur/new component.  The cur/new
>> directory is an internal implementation detail of Maildir (and a rather
>> broken one at that) and no more a part of the "folder" of a piece of
>> mail than its final file name component.  It's also the less obvious
>> user interface; if we require the cur/new component, we *will* get
>> people asking why their folder searches aren't working, but if we strip
>> the cur/new component, nobody will be surprised.
>>
>> I think the question is not whether we should strip cur/new, but when.
>> We've already defined a "_filename_is_in_maildir" test in
>> lib/message.cc, which we depend on for flag sync.  It's simple, but I
>> think this would be the right thing to use for consistency.
>
> I'd like to discuss some of the reasons I chose to include the cur/new
> components. Admittedly, none of them are very strong on their own, but
> all of them together tilted my opinion towards requiring them.
>
> The way I see it, notmuch supports maildir, but does not require it. In
> many ways the messages are just files somewhere in the directory
> hierarchy. There are only a few cases where it matters that there are
> cur/new/tmp directories within a directory.
>
> If you strip cur and new, it becomes impossible to distinguish between
> files in foo, foo/cur, and foo/new - and one of the reasons for changing
> folder: in the first place is to be able to better distinguish between
> folders.
>
> Apparently mutt presents the difference between messages in new and cur
> to its users (so I've been told; I've never really used mutt), and our
> integration with mutt lacks that distinction. We could fix that by
> requiring the cur/new components in folder: searches.
>
> Speaking of consistency, compare _filename_is_in_maildir() with
> _entries_resemble_maildir() in notmuch-new.c. What should the indexed
> folder: prefix be if there is not all of cur, new, and tmp? We will
> actually index files in tmp if cur or new is not present! What if the
> missing sibling directories are added (or existing ones removed) later?
> Where's the consistency compared to new.ignore config, which also
> matches the cur/new components if so desired? Or consistency with
> notmuch search --output=files?
>
> My conclusion was that requiring *all* filesystem folder components
> as-is is consistent, most versatile, agnostic to Maildir or Maildir++
> implementation details wrt directory naming or hierarchy, without
> difficult corner cases, simplest to implement, and unsurprising (once
> you understand the cur/new distinction).
>
> For *me* this is the more obvious user interface. And hey, I'm a user
> too.
>
> Perhaps we need to have two prefixes, one of which is the literal
> filesystem folder and another which hides the implementation details,
> like I mentioned in my mail to Peter [1]. But consider this: my proposed
> implementation does cover *all* use cases.

I challenge that with my use case: my mails are arranged as follows: 

head of contents of notmuch archive prior to my involvement:

$ find notmuch | head -5
notmuch
notmuch/6b
notmuch/6b/de820df0697ab2d235fbc8e32510d7
notmuch/6b/917afddb116a03c45371282be58388
notmuch/6b/10eb0bc1406f6767161f5883f328f7

head of contents of received mail

$ find notmuch | head -5
received
received/rawmail2
received/6b
received/6b/86a8937aac57721ad87f0e0e5fe6c3
received/6b/3278d6c4c1fe7604f1404bc09acff7

Interestingly find started with subdirectory '6b' in both cases...
-- anyway I have 0xff + 1 subdirectories in each mail directory and
$ md5sum received/6b/86a8937aac57721ad87f0e0e5fe6c3 outputs
6b86a8937aac57721ad87f0e0e5fe6c3 received/6b/86a8937aac57721ad87f0e0e5fe6c3

For me the current folder: works as I don't have collisions.

For me a folder: search which would just work as a prefix i.e. match
anything under given directory hierarchy would work best.

At the end it might be that I have to hack the search for my purposes;
more important/interesting thing is whether I need to use incompatible
database format :O


Tomi


>
> BR,
> Jani.
>
>
> [1] id:8761ppurfz.fsf@nikula.org