From: Ethan Glasser-Camp Date: Tue, 16 Oct 2012 00:13:01 +0000 (+2000) Subject: Re: [PATCH 3/3] new: don't read unchanged directories from disk X-Git-Url: http://git.tremily.us/gitweb.cgi?a=commitdiff_plain;h=7902c8d68568a75b45f2b2bec07f0cea596d341b;p=notmuch-archives.git Re: [PATCH 3/3] new: don't read unchanged directories from disk --- diff --git a/0c/d006bb6ee6dc76ede87689c236cbb59a949ab3 b/0c/d006bb6ee6dc76ede87689c236cbb59a949ab3 new file mode 100644 index 000000000..fcc03b1c3 --- /dev/null +++ b/0c/d006bb6ee6dc76ede87689c236cbb59a949ab3 @@ -0,0 +1,93 @@ +Return-Path: +X-Original-To: notmuch@notmuchmail.org +Delivered-To: notmuch@notmuchmail.org +Received: from localhost (localhost [127.0.0.1]) + by olra.theworths.org (Postfix) with ESMTP id CB42F431FB6 + for ; Mon, 15 Oct 2012 17:13:07 -0700 (PDT) +X-Virus-Scanned: Debian amavisd-new at olra.theworths.org +X-Spam-Flag: NO +X-Spam-Score: -0.799 +X-Spam-Level: +X-Spam-Status: No, score=-0.799 tagged_above=-999 required=5 + tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, + FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled +Received: from olra.theworths.org ([127.0.0.1]) + by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) + with ESMTP id yYOY5iwFZpo9 for ; + Mon, 15 Oct 2012 17:13:05 -0700 (PDT) +Received: from mail-qa0-f53.google.com (mail-qa0-f53.google.com + [209.85.216.53]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) + (No client certificate requested) + by olra.theworths.org (Postfix) with ESMTPS id B6660431FAE + for ; Mon, 15 Oct 2012 17:13:05 -0700 (PDT) +Received: by mail-qa0-f53.google.com with SMTP id s11so2163371qaa.5 + for ; Mon, 15 Oct 2012 17:13:05 -0700 (PDT) +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; + h=from:to:cc:subject:in-reply-to:references:user-agent:date + :message-id:mime-version:content-type; + bh=t8EmuOexxqL8YFO6aVX4zN4D0lw/eaVpkR1H7q3exH4=; + b=MoJZj9Dr+pBL9MQneIwxvkgyZs4RhAs+A6oaGoW4mZhA5nmE2HNVhWN2gB0g3q01pH + KVLCklvKSbcVyFA2m7G3CV17KDw6psHNlYlnH4TZKcDCHd1CPID0jiQ8cqEVK0n1KQKH + nFQeZlRTd5MXGbt0FZNQ6DfaJR1I4EKkmiKzGOxkZ3OnXEs/rxDCoovRqGpn0n+sDY66 + OdhKnkHerDRMjeKp8I3TXAh50OHA9bMfHADWQ8EdDqlyVHufGEc/3r+pQEOpOl0W/aMH + 9m/FK12r7R3T+T+8CJ9D9MPkoLVHmnKzxOSCLICTlTDEXXS+y0Aew2y+BqyOpNDVylw/ + 1FMg== +Received: by 10.224.107.5 with SMTP id z5mr23458461qao.41.1350346384972; + Mon, 15 Oct 2012 17:13:04 -0700 (PDT) +Received: from smtp.gmail.com (p70-80.acedsl.com. [66.114.70.80]) + by mx.google.com with ESMTPS id cz8sm11134483qab.21.2012.10.15.17.13.03 + (version=TLSv1/SSLv3 cipher=OTHER); + Mon, 15 Oct 2012 17:13:04 -0700 (PDT) +From: Ethan Glasser-Camp +To: Sascha Silbe , + notmuch +Subject: Re: [PATCH 3/3] new: don't read unchanged directories from disk +In-Reply-To: <1340555366-25891-4-git-send-email-sascha-pgp@silbe.org> +References: <1340555366-25891-1-git-send-email-sascha-pgp@silbe.org> + <1340555366-25891-4-git-send-email-sascha-pgp@silbe.org> +User-Agent: Notmuch/0.14+45~g6ea9330 (http://notmuchmail.org) Emacs/23.3.1 + (x86_64-pc-linux-gnu) +Date: Mon, 15 Oct 2012 20:13:01 -0400 +Message-ID: <87y5j78eb6.fsf@betacantrips.com> +MIME-Version: 1.0 +Content-Type: text/plain; charset=us-ascii +X-BeenThere: notmuch@notmuchmail.org +X-Mailman-Version: 2.1.13 +Precedence: list +List-Id: "Use and development of the notmuch mail system." + +List-Unsubscribe: , + +List-Archive: +List-Post: +List-Help: +List-Subscribe: , + +X-List-Received-Date: Tue, 16 Oct 2012 00:13:07 -0000 + +Sascha Silbe writes: + +> Previously, notmuch new listed all directories on disk, even if they +> were unchanged from the state recorded in the database. This could take +> a huge amount of time for large numbers of mails as it would list each +> individual mail. +> +> By iterating over the subdirectories recorded in the database we can +> avoid accessing the file system for each unchanged directory. If the +> modification time does not match we fall back to a full file system scan +> so new subdirectories will get picked up and scanned recursively. + +Hi! Just going through the patch queue. These patches look fine to me, +but I think architecturally, notmuch needs to decide what it wants to do +-- if it wants to optimize directory-rescanning, or if it wants to +provide a better way to just add files known to be new, and let mail +delivery hook into that system. I'd prefer the second, but I think it's +OK to merge working code that does the first in the meantime. The +perfect is the enemy of the good. + +Austin raised some concerns on this patch set that I'm not sure were +completely addressed. For my part, it seems intuitive that a stat() +would be faster than a scandir() and so I would have no problem with +merging this series. + +Ethan