From: Jani Nikula <jani@nikula.org>
To: Austin Clements <amdragon@MIT.EDU>
Subject: Re: [PATCH] lib: Save filenames for files detected as "not an email
	file" in the database.
In-Reply-To: <20120121234919.GM16740@mit.edu>
References: <1327096827-5760-1-git-send-email-amdragon@mit.edu>
	<87lip0acfy.fsf@nikula.org> <20120121234919.GM16740@mit.edu>
User-Agent: Notmuch/0.11+76~g1de742d (http://notmuchmail.org) Emacs/23.3.1
	(i686-pc-linux-gnu)
Date: Sun, 22 Jan 2012 14:19:04 +0200
Message-ID: <87pqecylon.fsf@nikula.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Cc: notmuch@notmuchmail.org
Precedence: list

On Sat, 21 Jan 2012 18:49:19 -0500, Austin Clements <amdragon@MIT.EDU> wrote:
> Quoth Jani Nikula on Jan 22 at  1:00 am:
> > On Fri, 20 Jan 2012 17:00:27 -0500, Austin Clements <amdragon@MIT.EDU> wrote:
> > > Later runs of "notmuch new" won't scan these files again and won't
> > > print warnings.
> > > 
> > > Various programs (Dovecot, in my case) store indexes and caches and
> > > such in the maildir.  Without this, notmuch persistently complains
> > > about such files.
> > 
> > Overall, sounds good and doing this automagically is nice. Superficially
> > the code looks sensible, but I didn't really dig into it. A few nasty
> > questions instead:
> > 
> > What happens if you delete a non-email file? Does the entry stay in the
> > database?
> 
> Phooey.  I thought this worked, but you're right that it doesn't (I
> even wrote a test for this, but the test was based on a false
> assumption).  Non-email files do get returned by the directory
> iterator, so without any changes, notmuch new will notice that they're
> gone.  What I missed is that it then uses
> notmuch_database_find_message_by_filename to find the "message" and
> remove the filename, which won't work since there's no message to
> find.
> 
> I'll have to think about this more.

Sorry about that...

This feature has considerable overlap with file/subdirectory exclusion,
most recently referred to in id:"20120122113212.GA7084@X200". I like the
way your approach is automatic, but doing it manually with configurable
exclusions has certain explicitness to it, and altogether avoids the
problems here, don't you think? There apparently also are people who
wouldn't want notmuch to index some valid email files for one reason or
another.

I haven't thought this through, but what if the exclude/ignore feature
had both the option to specify explicit files/subdirs (patterns like
.gitignore?) that are ignored, and some sort of "auto" option you could
enable to ignore all non-email files without warnings? This would
obviously all happen in the cli.

That probably does not make your thinking any easier, I'm afraid... but
perhaps it provides another angle.


BR,
Jani.


> 
> > What happens if you replace a non-email file with an email file?
> 
> It will not notice because notmuch new only inspects directory mtimes.
> This would require checking the mtimes of every non-email in the
> database on every notmuch new.
> 
> > Does it matter what happens above?
> > 
> > These are corner cases, but what remains in TODO suggests that it would
> > be difficult to debug and figure out if the above ever did happen to
> > someone.
> 
> Yes.  It's possible this needs to get a search syntax before it is
> acceptable for general use.
> 
> > BR,
> > Jani.