Re: [PATCH 3/3] new: don't read unchanged directories from disk

diff --git a/0c/d006bb6ee6dc76ede87689c236cbb59a949ab3 b/0c/d006bb6ee6dc76ede87689c236cbb59a949ab3
new file mode 100644 (file)
index 0000000..fcc03b1
--- /dev/null
+++ b/0c/d006bb6ee6dc76ede87689c236cbb59a949ab3
@@ -0,0 +1,93 @@
+Return-Path: <ethan.glasser.camp@gmail.com>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+       by olra.theworths.org (Postfix) with ESMTP id CB42F431FB6\r
+       for <notmuch@notmuchmail.org>; Mon, 15 Oct 2012 17:13:07 -0700 (PDT)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -0.799\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-0.799 tagged_above=-999 required=5\r
+       tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1,\r
+       FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+       by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+       with ESMTP id yYOY5iwFZpo9 for <notmuch@notmuchmail.org>;\r
+       Mon, 15 Oct 2012 17:13:05 -0700 (PDT)\r
+Received: from mail-qa0-f53.google.com (mail-qa0-f53.google.com\r
+       [209.85.216.53]) (using TLSv1 with cipher RC4-SHA (128/128 bits))\r
+       (No client certificate requested)\r
+       by olra.theworths.org (Postfix) with ESMTPS id B6660431FAE\r
+       for <notmuch@notmuchmail.org>; Mon, 15 Oct 2012 17:13:05 -0700 (PDT)\r
+Received: by mail-qa0-f53.google.com with SMTP id s11so2163371qaa.5\r
+       for <notmuch@notmuchmail.org>; Mon, 15 Oct 2012 17:13:05 -0700 (PDT)\r
+DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;\r
+       h=from:to:cc:subject:in-reply-to:references:user-agent:date\r
+       :message-id:mime-version:content-type;\r
+       bh=t8EmuOexxqL8YFO6aVX4zN4D0lw/eaVpkR1H7q3exH4=;\r
+       b=MoJZj9Dr+pBL9MQneIwxvkgyZs4RhAs+A6oaGoW4mZhA5nmE2HNVhWN2gB0g3q01pH\r
+       KVLCklvKSbcVyFA2m7G3CV17KDw6psHNlYlnH4TZKcDCHd1CPID0jiQ8cqEVK0n1KQKH\r
+       nFQeZlRTd5MXGbt0FZNQ6DfaJR1I4EKkmiKzGOxkZ3OnXEs/rxDCoovRqGpn0n+sDY66\r
+       OdhKnkHerDRMjeKp8I3TXAh50OHA9bMfHADWQ8EdDqlyVHufGEc/3r+pQEOpOl0W/aMH\r
+       9m/FK12r7R3T+T+8CJ9D9MPkoLVHmnKzxOSCLICTlTDEXXS+y0Aew2y+BqyOpNDVylw/\r
+       1FMg==\r
+Received: by 10.224.107.5 with SMTP id z5mr23458461qao.41.1350346384972;\r
+       Mon, 15 Oct 2012 17:13:04 -0700 (PDT)\r
+Received: from smtp.gmail.com (p70-80.acedsl.com. [66.114.70.80])\r
+       by mx.google.com with ESMTPS id cz8sm11134483qab.21.2012.10.15.17.13.03\r
+       (version=TLSv1/SSLv3 cipher=OTHER);\r
+       Mon, 15 Oct 2012 17:13:04 -0700 (PDT)\r
+From: Ethan Glasser-Camp <ethan.glasser.camp@gmail.com>\r
+To: Sascha Silbe <sascha-ml-reply-to-2012-3@silbe.org>,\r
+       notmuch <notmuch@notmuchmail.org>\r
+Subject: Re: [PATCH 3/3] new: don't read unchanged directories from disk\r
+In-Reply-To: <1340555366-25891-4-git-send-email-sascha-pgp@silbe.org>\r
+References: <1340555366-25891-1-git-send-email-sascha-pgp@silbe.org>\r
+       <1340555366-25891-4-git-send-email-sascha-pgp@silbe.org>\r
+User-Agent: Notmuch/0.14+45~g6ea9330 (http://notmuchmail.org) Emacs/23.3.1\r
+       (x86_64-pc-linux-gnu)\r
+Date: Mon, 15 Oct 2012 20:13:01 -0400\r
+Message-ID: <87y5j78eb6.fsf@betacantrips.com>\r
+MIME-Version: 1.0\r
+Content-Type: text/plain; charset=us-ascii\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+       <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Tue, 16 Oct 2012 00:13:07 -0000\r
+\r
+Sascha Silbe <sascha-pgp@silbe.org> writes:\r
+\r
+> Previously, notmuch new listed all directories on disk, even if they\r
+> were unchanged from the state recorded in the database. This could take\r
+> a huge amount of time for large numbers of mails as it would list each\r
+> individual mail.\r
+>\r
+> By iterating over the subdirectories recorded in the database we can\r
+> avoid accessing the file system for each unchanged directory. If the\r
+> modification time does not match we fall back to a full file system scan\r
+> so new subdirectories will get picked up and scanned recursively.\r
+\r
+Hi! Just going through the patch queue. These patches look fine to me,\r
+but I think architecturally, notmuch needs to decide what it wants to do\r
+-- if it wants to optimize directory-rescanning, or if it wants to\r
+provide a better way to just add files known to be new, and let mail\r
+delivery hook into that system. I'd prefer the second, but I think it's\r
+OK to merge working code that does the first in the meantime. The\r
+perfect is the enemy of the good.\r
+\r
+Austin raised some concerns on this patch set that I'm not sure were\r
+completely addressed. For my part, it seems intuitive that a stat()\r
+would be faster than a scandir() and so I would have no problem with\r
+merging this series.\r
+\r
+Ethan\r