From: Jani Nikula <jani@nikula.org>
To: Austin Clements <amdragon@mit.edu>, Thomas Schwinge <thomas@schwinge.name>
Cc: notmuch@notmuchmail.org
Subject: Re: [PATCH] Repeatability when copying a whole directory into a
 new	one.
In-Reply-To:
 <CAH-f9WuE3quh1jneP1Zj4+xB7r+AXwKbX0QfYvEwOQEKFG1F8Q@mail.gmail.com>
References: <1317338806-7414-1-git-send-email-thomas@schwinge.name>
 <CAH-f9WuE3quh1jneP1Zj4+xB7r+AXwKbX0QfYvEwOQEKFG1F8Q@mail.gmail.com>
User-Agent: Notmuch/0.21+109~g54aeab1 (http://notmuchmail.org) Emacs/24.4.1
 (x86_64-pc-linux-gnu)
Date: Wed, 13 Apr 2016 22:50:03 +0300
Message-ID: <87a8kxwaok.fsf@nikula.org>
MIME-Version: 1.0
Content-Type: text/plain
Precedence: list

On Sat, 05 Nov 2011, Austin Clements <amdragon@mit.edu> wrote:
> On Thu, Sep 29, 2011 at 7:26 PM, Thomas Schwinge <thomas@schwinge.name> wrote:
>> This new test currently fails -- but it shouldn't.
>> ---
>>
>> Hi!
>>
>> I found this while manually copying directories and running notmuch new.
>>
>> Am I just too sleepy at this time, or is it another DB vs. directory
>> mtime issue?
>
> Nice catch.  I haven't verified this, but I believe the problem is
> that notmuch never deletes directory documents.  In fact, there isn't
> even an API to do so.  Even though it detects the deleted directory
> and deletes all messages under it, the directory document sticks
> around.  When the directory comes back, notmuch finds the old
> directory document with the old directory mtime and thinks it doesn't
> have to rescan the directory because the cp -a reproduced the same
> mtime the directory used to have.
>
> So, I guess part of the answer is "don't cp -a" because that mucks
> with mtimes and mtimes are all notmuch has to go by.  But that's no
> excuse for not removing the directory documents when the directory is
> removed.
>
> Fixing this is slightly tricky.  I feel like there *shouldn't* be an
> API to simply remove a directory document because that would let you
> violate database consistency.  Instead, I think the API should
> recursively remove everything under the removed directory, exactly
> like what notmuch-new.c:_remove_directory does right now (plus
> removing the directory documents).  But _remove_directory depends on
> remove_filename, which currently has notmuch-new-specific logic in it.
>  I feel like there must be a nice solution to this, and I'm just not
> thinking of it.

Stumbled upon this one while checking old bug reports. Maybe the fix in

commit e26d99dc7b02f33299c281f97a13deaef802bc7a
Author: Jani Nikula <jani@nikula.org>
Date:   Fri Sep 25 23:48:46 2015 +0300

    cli: delete directory documents on directory removal

is inelegant, as it adds the API to remove directory document, but it's
there now. I was unaware of this bug report and your analysis at the
time.

BR,
Jani.