1 Return-Path: <tomi.ollila@iki.fi>
\r
2 X-Original-To: notmuch@notmuchmail.org
\r
3 Delivered-To: notmuch@notmuchmail.org
\r
4 Received: from localhost (localhost [127.0.0.1])
\r
5 by olra.theworths.org (Postfix) with ESMTP id 61830431FC2
\r
6 for <notmuch@notmuchmail.org>; Fri, 25 Oct 2013 04:46:39 -0700 (PDT)
\r
7 X-Virus-Scanned: Debian amavisd-new at olra.theworths.org
\r
11 X-Spam-Status: No, score=0 tagged_above=-999 required=5 tests=[none]
\r
13 Received: from olra.theworths.org ([127.0.0.1])
\r
14 by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)
\r
15 with ESMTP id u38zQyKnXA4L for <notmuch@notmuchmail.org>;
\r
16 Fri, 25 Oct 2013 04:46:31 -0700 (PDT)
\r
17 Received: from guru.guru-group.fi (guru.guru-group.fi [46.183.73.34])
\r
18 by olra.theworths.org (Postfix) with ESMTP id CF22C431FB6
\r
19 for <notmuch@notmuchmail.org>; Fri, 25 Oct 2013 04:46:30 -0700 (PDT)
\r
20 Received: from guru.guru-group.fi (localhost [IPv6:::1])
\r
21 by guru.guru-group.fi (Postfix) with ESMTP id 364EB100217;
\r
22 Fri, 25 Oct 2013 14:46:21 +0300 (EEST)
\r
23 From: Tomi Ollila <tomi.ollila@iki.fi>
\r
24 To: Austin Clements <amdragon@MIT.EDU>, notmuch@notmuchmail.org
\r
25 Subject: Re: [PATCH v2] new: Don't scan unchanged directories with no
\r
27 In-Reply-To: <1382650739-12438-1-git-send-email-amdragon@mit.edu>
\r
28 References: <20131024210837.GH20337@mit.edu>
\r
29 <1382650739-12438-1-git-send-email-amdragon@mit.edu>
\r
30 User-Agent: Notmuch/0.16+115~g11c2ff5 (http://notmuchmail.org) Emacs/24.3.1
\r
31 (x86_64-unknown-linux-gnu)
\r
32 X-Face: HhBM'cA~<r"^Xv\KRN0P{vn'Y"Kd;zg_y3S[4)KSN~s?O\"QPoL
\r
33 $[Xv_BD:i/F$WiEWax}R(MPS`^UaptOGD`*/=@\1lKoVa9tnrg0TW?"r7aRtgk[F
\r
34 !)g;OY^,BjTbr)Np:%c_o'jj,Z
\r
35 Date: Fri, 25 Oct 2013 14:46:21 +0300
\r
36 Message-ID: <m28uxhefoy.fsf@guru.guru-group.fi>
\r
38 Content-Type: text/plain
\r
39 X-BeenThere: notmuch@notmuchmail.org
\r
40 X-Mailman-Version: 2.1.13
\r
42 List-Id: "Use and development of the notmuch mail system."
\r
43 <notmuch.notmuchmail.org>
\r
44 List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,
\r
45 <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>
\r
46 List-Archive: <http://notmuchmail.org/pipermail/notmuch>
\r
47 List-Post: <mailto:notmuch@notmuchmail.org>
\r
48 List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>
\r
49 List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,
\r
50 <mailto:notmuch-request@notmuchmail.org?subject=subscribe>
\r
51 X-List-Received-Date: Fri, 25 Oct 2013 11:46:39 -0000
\r
53 On Fri, Oct 25 2013, Austin Clements <amdragon@MIT.EDU> wrote:
\r
55 > This can substantially reduce the cost of notmuch new in some
\r
56 > situations, such as when the file system cache is cold or when the
\r
57 > Maildir is on NFS.
\r
60 LGTM. The creation and destruction of child directories happens
\r
61 only if there are symlinks to directories in otherwise leaf directories.
\r
66 > This should fix the problem with directories containing symlinks to
\r
67 > other directories, but no actual sub-directories.
\r
69 > notmuch-new.c | 29 +++++++++++++++++++++++++++++
\r
70 > 1 file changed, 29 insertions(+)
\r
72 > diff --git a/notmuch-new.c b/notmuch-new.c
\r
73 > index faa33f1..ba05cb4 100644
\r
74 > --- a/notmuch-new.c
\r
75 > +++ b/notmuch-new.c
\r
76 > @@ -323,6 +323,35 @@ add_files (notmuch_database_t *notmuch,
\r
78 > db_mtime = directory ? notmuch_directory_get_mtime (directory) : 0;
\r
80 > + /* If the directory is unchanged from our last scan and has no
\r
81 > + * sub-directories, then return without scanning it at all. In
\r
82 > + * some situations, skipping the scan can substantially reduce the
\r
83 > + * cost of notmuch new, especially since the huge numbers of files
\r
84 > + * in Maildirs make scans expensive, but all files live in leaf
\r
87 > + * To check for sub-directories, we borrow a trick from find,
\r
88 > + * kpathsea, and many other UNIX tools: since a directory's link
\r
89 > + * count is the number of sub-directories (specifically, their
\r
90 > + * '..' entries) plus 2 (the link from the parent and the link for
\r
91 > + * '.'). This check is safe even on weird file systems, since
\r
92 > + * file systems that can't compute this will return 0 or 1. This
\r
93 > + * is safe even on *really* weird file systems like HFS+ that
\r
94 > + * mistakenly return the total number of directory entries, since
\r
95 > + * that only inflates the count beyond 2.
\r
97 > + if (directory && fs_mtime == db_mtime && st.st_nlink == 2) {
\r
98 > + /* There's one catch: pass 1 below considers symlinks to
\r
99 > + * directories to be directories, but these don't increase the
\r
100 > + * file system link count. So, only bail early if the
\r
101 > + * database agrees that there are no sub-directories. */
\r
102 > + db_subdirs = notmuch_directory_get_child_directories (directory);
\r
103 > + if (!notmuch_filenames_valid (db_subdirs))
\r
105 > + notmuch_filenames_destroy (db_subdirs);
\r
106 > + db_subdirs = NULL;
\r
109 > /* If the database knows about this directory, then we sort based
\r
110 > * on strcmp to match the database sorting. Otherwise, we can do
\r
111 > * inode-based sorting for faster filesystem operation. */
\r
115 > _______________________________________________
\r
116 > notmuch mailing list
\r
117 > notmuch@notmuchmail.org
\r
118 > http://notmuchmail.org/mailman/listinfo/notmuch
\r