Re: [PATCH] Repeatability when copying a whole directory into a new one.

diff --git a/c6/0ed17349838f93c3b123b9dbbaba8427b0a171 b/c6/0ed17349838f93c3b123b9dbbaba8427b0a171
new file mode 100644 (file)
index 0000000..dd1d4ca
--- /dev/null
+++ b/c6/0ed17349838f93c3b123b9dbbaba8427b0a171
@@ -0,0 +1,133 @@
+Return-Path: <jani@nikula.org>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+ by arlo.cworth.org (Postfix) with ESMTP id B2AF76DE014D\r
+ for <notmuch@notmuchmail.org>; Wed, 13 Apr 2016 12:51:28 -0700 (PDT)\r
+X-Virus-Scanned: Debian amavisd-new at cworth.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -0.555\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-0.555 tagged_above=-999 required=5 tests=[AWL=0.165,\r
+  DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_LOW=-0.7,\r
+ RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01] autolearn=disabled\r
+Received: from arlo.cworth.org ([127.0.0.1])\r
+ by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024)\r
+ with ESMTP id 7vXhiEa4Y3LR for <notmuch@notmuchmail.org>;\r
+ Wed, 13 Apr 2016 12:51:20 -0700 (PDT)\r
+Received: from mail-wm0-f65.google.com (mail-wm0-f65.google.com\r
+ [74.125.82.65]) by arlo.cworth.org (Postfix) with ESMTPS id 45BDA6DE0127 for\r
+ <notmuch@notmuchmail.org>; Wed, 13 Apr 2016 12:51:20 -0700 (PDT)\r
+Received: by mail-wm0-f65.google.com with SMTP id n3so17013575wmn.1\r
+ for <notmuch@notmuchmail.org>; Wed, 13 Apr 2016 12:51:20 -0700 (PDT)\r
+DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;\r
+ d=nikula-org.20150623.gappssmtp.com; s=20150623;\r
+ h=from:to:cc:subject:in-reply-to:references:user-agent:date\r
+ :message-id:mime-version;\r
+ bh=c2tCIWelNfH9xSJvFv9XV7IBSNeLEgmKYGVOY5GALW0=;\r
+ b=VgbTxdYUrNeUQ2iN0HhrYMH+bAmDY4GXjruTYqeNLpO87XCMoaHPr/EbYAvOpyDkb0\r
+ dg8kjYodq7UHl9erKZUSdEA2C1LL5ZPHBtFjtsPPU3PiVx/09FkPg+0fPtO8pFvPf3e/\r
+ jl4BNeDo4RT9jwq+gfWecoIy7f+Ys1FOY77UIoyYKIn/zxwsPcRIBdFYAoPI/4042M1b\r
+ 6jSWGTlJHyENNQr7Immi+EB5v/kppYRldsG/WpwFs1jnzk10qjDuRr0UtyBHvzwA9e3T\r
+ wb8bn+506QHPz0/PO2gwPTl/IR58JhNY83/NEhIUcHJk71fx0JdoVegOwmQeBQ+NvDPP\r
+ y9gg==\r
+X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;\r
+ d=1e100.net; s=20130820;\r
+ h=x-gm-message-state:from:to:cc:subject:in-reply-to:references\r
+ :user-agent:date:message-id:mime-version;\r
+ bh=c2tCIWelNfH9xSJvFv9XV7IBSNeLEgmKYGVOY5GALW0=;\r
+ b=BRVdbbb7UkIYTiib4t/4yoko+Di3S8dU7vmQ6SFJGhdrv4x6V04y4D1MLgZcJt9yHa\r
+ /q5e+yYgPSYMjlBOu9Wl3C+OFuBLmrId1kK6L5N/ENPVXGDVs1uRah4oBU6T/hD36OSc\r
+ iG7S+CGE/seVYOV7eEkRov8Aw9q54TM3q0VfYv0hrng7OwjtnnEjoEqOUzpBmyFCJGaO\r
+ ds8s2kkoTfqK6T4sfUCm4vE7MvWjrVdc1fkQkRgz7XSmdhLvoScTcj2VAe3dRawQ+bgV\r
+ KrZ+5RhXAa8jzg/JfgPMio98PAg6IFLF76HgaqS4fOAUdM8H+jGnQJ5P6NnT9G35TuWe\r
+ 1jqQ==\r
+X-Gm-Message-State:\r
+ AOPr4FVka003pzpKuF02I/bVjcc7oek0L4y6MEBZk0XGefAPpWMeY3tTjkd6QM6kD1mD1g==\r
+X-Received: by 10.194.205.167 with SMTP id lh7mr11723858wjc.30.1460577078723; \r
+ Wed, 13 Apr 2016 12:51:18 -0700 (PDT)\r
+Received: from localhost (mobile-access-bcee7f-102.dhcp.inet.fi.\r
+ [188.238.127.102])\r
+ by smtp.gmail.com with ESMTPSA id g78sm29746887wme.21.2016.04.13.12.51.17\r
+ (version=TLSv1/SSLv3 cipher=OTHER);\r
+ Wed, 13 Apr 2016 12:51:17 -0700 (PDT)\r
+From: Jani Nikula <jani@nikula.org>\r
+To: Austin Clements <amdragon@mit.edu>, Thomas Schwinge <thomas@schwinge.name>\r
+Cc: notmuch@notmuchmail.org\r
+Subject: Re: [PATCH] Repeatability when copying a whole directory into a\r
+ new   one.\r
+In-Reply-To:\r
+ <CAH-f9WuE3quh1jneP1Zj4+xB7r+AXwKbX0QfYvEwOQEKFG1F8Q@mail.gmail.com>\r
+References: <1317338806-7414-1-git-send-email-thomas@schwinge.name>\r
+ <CAH-f9WuE3quh1jneP1Zj4+xB7r+AXwKbX0QfYvEwOQEKFG1F8Q@mail.gmail.com>\r
+User-Agent: Notmuch/0.21+109~g54aeab1 (http://notmuchmail.org) Emacs/24.4.1\r
+ (x86_64-pc-linux-gnu)\r
+Date: Wed, 13 Apr 2016 22:50:03 +0300\r
+Message-ID: <87a8kxwaok.fsf@nikula.org>\r
+MIME-Version: 1.0\r
+Content-Type: text/plain\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.20\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+ <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <https://notmuchmail.org/mailman/options/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch/>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <https://notmuchmail.org/mailman/listinfo/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Wed, 13 Apr 2016 19:51:28 -0000\r
+\r
+On Sat, 05 Nov 2011, Austin Clements <amdragon@mit.edu> wrote:\r
+> On Thu, Sep 29, 2011 at 7:26 PM, Thomas Schwinge <thomas@schwinge.name> wrote:\r
+>> This new test currently fails -- but it shouldn't.\r
+>> ---\r
+>>\r
+>> Hi!\r
+>>\r
+>> I found this while manually copying directories and running notmuch new.\r
+>>\r
+>> Am I just too sleepy at this time, or is it another DB vs. directory\r
+>> mtime issue?\r
+>\r
+> Nice catch.  I haven't verified this, but I believe the problem is\r
+> that notmuch never deletes directory documents.  In fact, there isn't\r
+> even an API to do so.  Even though it detects the deleted directory\r
+> and deletes all messages under it, the directory document sticks\r
+> around.  When the directory comes back, notmuch finds the old\r
+> directory document with the old directory mtime and thinks it doesn't\r
+> have to rescan the directory because the cp -a reproduced the same\r
+> mtime the directory used to have.\r
+>\r
+> So, I guess part of the answer is "don't cp -a" because that mucks\r
+> with mtimes and mtimes are all notmuch has to go by.  But that's no\r
+> excuse for not removing the directory documents when the directory is\r
+> removed.\r
+>\r
+> Fixing this is slightly tricky.  I feel like there *shouldn't* be an\r
+> API to simply remove a directory document because that would let you\r
+> violate database consistency.  Instead, I think the API should\r
+> recursively remove everything under the removed directory, exactly\r
+> like what notmuch-new.c:_remove_directory does right now (plus\r
+> removing the directory documents).  But _remove_directory depends on\r
+> remove_filename, which currently has notmuch-new-specific logic in it.\r
+>  I feel like there must be a nice solution to this, and I'm just not\r
+> thinking of it.\r
+\r
+Stumbled upon this one while checking old bug reports. Maybe the fix in\r
+\r
+commit e26d99dc7b02f33299c281f97a13deaef802bc7a\r
+Author: Jani Nikula <jani@nikula.org>\r
+Date:   Fri Sep 25 23:48:46 2015 +0300\r
+\r
+    cli: delete directory documents on directory removal\r
+\r
+is inelegant, as it adds the API to remove directory document, but it's\r
+there now. I was unaware of this bug report and your analysis at the\r
+time.\r
+\r
+BR,\r
+Jani.\r
+\r