From: Jani Nikula Date: Wed, 13 Apr 2016 19:50:03 +0000 (+0300) Subject: Re: [PATCH] Repeatability when copying a whole directory into a new one. X-Git-Url: http://git.tremily.us/?a=commitdiff_plain;h=e94390a11914b4fcf7ac112729f6b09f052f6dae;p=notmuch-archives.git Re: [PATCH] Repeatability when copying a whole directory into a new one. --- diff --git a/c6/0ed17349838f93c3b123b9dbbaba8427b0a171 b/c6/0ed17349838f93c3b123b9dbbaba8427b0a171 new file mode 100644 index 000000000..dd1d4ca41 --- /dev/null +++ b/c6/0ed17349838f93c3b123b9dbbaba8427b0a171 @@ -0,0 +1,133 @@ +Return-Path: +X-Original-To: notmuch@notmuchmail.org +Delivered-To: notmuch@notmuchmail.org +Received: from localhost (localhost [127.0.0.1]) + by arlo.cworth.org (Postfix) with ESMTP id B2AF76DE014D + for ; Wed, 13 Apr 2016 12:51:28 -0700 (PDT) +X-Virus-Scanned: Debian amavisd-new at cworth.org +X-Spam-Flag: NO +X-Spam-Score: -0.555 +X-Spam-Level: +X-Spam-Status: No, score=-0.555 tagged_above=-999 required=5 tests=[AWL=0.165, + DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_LOW=-0.7, + RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01] autolearn=disabled +Received: from arlo.cworth.org ([127.0.0.1]) + by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024) + with ESMTP id 7vXhiEa4Y3LR for ; + Wed, 13 Apr 2016 12:51:20 -0700 (PDT) +Received: from mail-wm0-f65.google.com (mail-wm0-f65.google.com + [74.125.82.65]) by arlo.cworth.org (Postfix) with ESMTPS id 45BDA6DE0127 for + ; Wed, 13 Apr 2016 12:51:20 -0700 (PDT) +Received: by mail-wm0-f65.google.com with SMTP id n3so17013575wmn.1 + for ; Wed, 13 Apr 2016 12:51:20 -0700 (PDT) +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; + d=nikula-org.20150623.gappssmtp.com; s=20150623; + h=from:to:cc:subject:in-reply-to:references:user-agent:date + :message-id:mime-version; + bh=c2tCIWelNfH9xSJvFv9XV7IBSNeLEgmKYGVOY5GALW0=; + b=VgbTxdYUrNeUQ2iN0HhrYMH+bAmDY4GXjruTYqeNLpO87XCMoaHPr/EbYAvOpyDkb0 + dg8kjYodq7UHl9erKZUSdEA2C1LL5ZPHBtFjtsPPU3PiVx/09FkPg+0fPtO8pFvPf3e/ + jl4BNeDo4RT9jwq+gfWecoIy7f+Ys1FOY77UIoyYKIn/zxwsPcRIBdFYAoPI/4042M1b + 6jSWGTlJHyENNQr7Immi+EB5v/kppYRldsG/WpwFs1jnzk10qjDuRr0UtyBHvzwA9e3T + wb8bn+506QHPz0/PO2gwPTl/IR58JhNY83/NEhIUcHJk71fx0JdoVegOwmQeBQ+NvDPP + y9gg== +X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; + d=1e100.net; s=20130820; + h=x-gm-message-state:from:to:cc:subject:in-reply-to:references + :user-agent:date:message-id:mime-version; + bh=c2tCIWelNfH9xSJvFv9XV7IBSNeLEgmKYGVOY5GALW0=; + b=BRVdbbb7UkIYTiib4t/4yoko+Di3S8dU7vmQ6SFJGhdrv4x6V04y4D1MLgZcJt9yHa + /q5e+yYgPSYMjlBOu9Wl3C+OFuBLmrId1kK6L5N/ENPVXGDVs1uRah4oBU6T/hD36OSc + iG7S+CGE/seVYOV7eEkRov8Aw9q54TM3q0VfYv0hrng7OwjtnnEjoEqOUzpBmyFCJGaO + ds8s2kkoTfqK6T4sfUCm4vE7MvWjrVdc1fkQkRgz7XSmdhLvoScTcj2VAe3dRawQ+bgV + KrZ+5RhXAa8jzg/JfgPMio98PAg6IFLF76HgaqS4fOAUdM8H+jGnQJ5P6NnT9G35TuWe + 1jqQ== +X-Gm-Message-State: + AOPr4FVka003pzpKuF02I/bVjcc7oek0L4y6MEBZk0XGefAPpWMeY3tTjkd6QM6kD1mD1g== +X-Received: by 10.194.205.167 with SMTP id lh7mr11723858wjc.30.1460577078723; + Wed, 13 Apr 2016 12:51:18 -0700 (PDT) +Received: from localhost (mobile-access-bcee7f-102.dhcp.inet.fi. + [188.238.127.102]) + by smtp.gmail.com with ESMTPSA id g78sm29746887wme.21.2016.04.13.12.51.17 + (version=TLSv1/SSLv3 cipher=OTHER); + Wed, 13 Apr 2016 12:51:17 -0700 (PDT) +From: Jani Nikula +To: Austin Clements , Thomas Schwinge +Cc: notmuch@notmuchmail.org +Subject: Re: [PATCH] Repeatability when copying a whole directory into a + new one. +In-Reply-To: + +References: <1317338806-7414-1-git-send-email-thomas@schwinge.name> + +User-Agent: Notmuch/0.21+109~g54aeab1 (http://notmuchmail.org) Emacs/24.4.1 + (x86_64-pc-linux-gnu) +Date: Wed, 13 Apr 2016 22:50:03 +0300 +Message-ID: <87a8kxwaok.fsf@nikula.org> +MIME-Version: 1.0 +Content-Type: text/plain +X-BeenThere: notmuch@notmuchmail.org +X-Mailman-Version: 2.1.20 +Precedence: list +List-Id: "Use and development of the notmuch mail system." + +List-Unsubscribe: , + +List-Archive: +List-Post: +List-Help: +List-Subscribe: , + +X-List-Received-Date: Wed, 13 Apr 2016 19:51:28 -0000 + +On Sat, 05 Nov 2011, Austin Clements wrote: +> On Thu, Sep 29, 2011 at 7:26 PM, Thomas Schwinge wrote: +>> This new test currently fails -- but it shouldn't. +>> --- +>> +>> Hi! +>> +>> I found this while manually copying directories and running notmuch new. +>> +>> Am I just too sleepy at this time, or is it another DB vs. directory +>> mtime issue? +> +> Nice catch. I haven't verified this, but I believe the problem is +> that notmuch never deletes directory documents. In fact, there isn't +> even an API to do so. Even though it detects the deleted directory +> and deletes all messages under it, the directory document sticks +> around. When the directory comes back, notmuch finds the old +> directory document with the old directory mtime and thinks it doesn't +> have to rescan the directory because the cp -a reproduced the same +> mtime the directory used to have. +> +> So, I guess part of the answer is "don't cp -a" because that mucks +> with mtimes and mtimes are all notmuch has to go by. But that's no +> excuse for not removing the directory documents when the directory is +> removed. +> +> Fixing this is slightly tricky. I feel like there *shouldn't* be an +> API to simply remove a directory document because that would let you +> violate database consistency. Instead, I think the API should +> recursively remove everything under the removed directory, exactly +> like what notmuch-new.c:_remove_directory does right now (plus +> removing the directory documents). But _remove_directory depends on +> remove_filename, which currently has notmuch-new-specific logic in it. +> I feel like there must be a nice solution to this, and I'm just not +> thinking of it. + +Stumbled upon this one while checking old bug reports. Maybe the fix in + +commit e26d99dc7b02f33299c281f97a13deaef802bc7a +Author: Jani Nikula +Date: Fri Sep 25 23:48:46 2015 +0300 + + cli: delete directory documents on directory removal + +is inelegant, as it adds the API to remove directory document, but it's +there now. I was unaware of this bug report and your analysis at the +time. + +BR, +Jani. +