1 Return-Path: <cworth@cworth.org>
\r
2 X-Original-To: notmuch@notmuchmail.org
\r
3 Delivered-To: notmuch@notmuchmail.org
\r
4 Received: from localhost (localhost [127.0.0.1])
\r
5 by olra.theworths.org (Postfix) with ESMTP id AEB17431FBD;
\r
6 Thu, 3 Dec 2009 16:45:27 -0800 (PST)
\r
7 X-Virus-Scanned: Debian amavisd-new at olra.theworths.org
\r
8 Received: from olra.theworths.org ([127.0.0.1])
\r
9 by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)
\r
10 with ESMTP id bc83K61DuFi3; Thu, 3 Dec 2009 16:45:27 -0800 (PST)
\r
11 Received: from yoom.home.cworth.org (localhost [127.0.0.1])
\r
12 by olra.theworths.org (Postfix) with ESMTP id 42DB5431FAE;
\r
13 Thu, 3 Dec 2009 16:45:24 -0800 (PST)
\r
14 Received: by yoom.home.cworth.org (Postfix, from userid 1000)
\r
15 id CCBA72542FB; Thu, 3 Dec 2009 16:45:22 -0800 (PST)
\r
16 From: Carl Worth <cworth@cworth.org>
\r
17 To: Mikhail Gusarov <dottedmag@dottedmag.net>, notmuch@notmuchmail.org
\r
18 In-Reply-To: <1259788526-14205-1-git-send-email-dottedmag@dottedmag.net>
\r
19 References: <1259267025-28733-1-git-send-email-dottedmag@dottedmag.net>
\r
20 <1259788526-14205-1-git-send-email-dottedmag@dottedmag.net>
\r
21 Date: Thu, 03 Dec 2009 16:45:22 -0800
\r
22 Message-ID: <87zl5zfty5.fsf@yoom.home.cworth.org>
\r
24 Content-Type: multipart/signed; boundary="=-=-=";
\r
25 micalg=pgp-sha1; protocol="application/pgp-signature"
\r
26 Subject: Re: [notmuch] [PATCH (rebased)] Handle message renames in mail spool
\r
27 X-BeenThere: notmuch@notmuchmail.org
\r
28 X-Mailman-Version: 2.1.12
\r
30 List-Id: "Use and development of the notmuch mail system."
\r
31 <notmuch.notmuchmail.org>
\r
32 List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,
\r
33 <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>
\r
34 List-Archive: <http://notmuchmail.org/pipermail/notmuch>
\r
35 List-Post: <mailto:notmuch@notmuchmail.org>
\r
36 List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>
\r
37 List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,
\r
38 <mailto:notmuch-request@notmuchmail.org?subject=subscribe>
\r
39 X-List-Received-Date: Fri, 04 Dec 2009 00:45:27 -0000
\r
43 On Thu, 3 Dec 2009 03:15:26 +0600, Mikhail Gusarov <dottedmag@dottedmag.net> wrote:
\r
44 > In order to handle message renames the following changes were deemed
\r
49 Thanks for contributing this patch (twice!). I think if I had gotten to
\r
50 it sooner, I probably would have committed it. But now...
\r
52 > * Mtime check on individual files was disabled. As files may be moved around
\r
53 > without changing their mtime, it's necessary to parse them even if they appear
\r
54 > old in case old message was moved. mtime check on directories was kept as moving
\r
55 > files changes mtime of directory.
\r
57 That sounds pretty harsh. I'm having to do a lot of stat() calls already
\r
58 when new mail arrives. Having to also parse the message ID out of
\r
59 (roughly, for me) 10000 files every time sounds pretty rough. Fortunately...
\r
61 > Note that after applying this patch notmuch still does not handle copying files
\r
62 > (which is harmless, database will point to the last copy of message found during
\r
63 > 'notmuch new') and deleting files (which is more serious, as dangling entries
\r
64 > will show up in searches).
\r
66 Today, Keith and designed an interface that will support addition,
\r
67 copying, rename, and deletion of files. And it will be faster than the
\r
68 existing code with its mtime heuristics.
\r
70 The complete design is on Keith's laptop right now, and hopefully he'll
\r
71 appear soon with an implementation. Basically, there are only two new
\r
72 functions needed in the library (if we got the design right):
\r
75 notmuch_database_read_directory (notmuch_database_t *database,
\r
79 notmuch_message_remove_filename (notmuch_message_t *message,
\r
80 const char *filename);
\r
82 The notmuch_directory_t object will be used in place of the current
\r
83 notmuch_database_get_timestamp call in notmuch-new.c. In addition to the
\r
84 mtime that we currently read from the database, it will provide a list
\r
85 of all directories and files (along with message IDs) known to the
\r
86 database for a particular path. So notmuch-new can then quickly compare
\r
87 the results of scandir with this notmuch_directory_t object and then
\r
88 call notmuch_database_add_message and notmuch_message_remove_filename as
\r
91 I'm leaving out details about how to ensure we don't delete a message
\r
92 too soon if it's actually a rename that will be seen as an added file
\r
93 later in the scan. Obviously the implementation will need to deal with
\r
94 that, (either with an additional library call for "I'm done adding
\r
95 files, go ahead and delete dangling messages", or by postponing all
\r
96 calls to remove_filename until later).
\r
98 Oh, and one idea is to do deletion by dropping all indexed terms, but
\r
99 saving the message ID and any tags in the database. That's small and is
\r
100 the only precious data, so might be worth holding onto "just in case".
\r
102 Anyway, I think we'll see code for that soon, so I'm not planning to
\r
103 commit the offered patch. But people really needing renames might want
\r
104 to use it for now, (and live with any performance implications it
\r
110 Content-Type: application/pgp-signature
\r
112 -----BEGIN PGP SIGNATURE-----
\r
113 Version: GnuPG v1.4.10 (GNU/Linux)
\r
115 iD8DBQFLGFui6JDdNq8qSWgRArJgAJ9/r/qgcmTOXv9DZAu1y0uTAJhoigCgpmnN
\r
116 /uZ6RHSHFN2Ou8YPb4XSZ/I=
\r
118 -----END PGP SIGNATURE-----
\r
projects / notmuch-archives.git / blob