1 Return-Path: <amdragon@mit.edu>
\r
2 X-Original-To: notmuch@notmuchmail.org
\r
3 Delivered-To: notmuch@notmuchmail.org
\r
4 Received: from localhost (localhost [127.0.0.1])
\r
5 by olra.theworths.org (Postfix) with ESMTP id 123C4431FB6
\r
6 for <notmuch@notmuchmail.org>; Sat, 8 Mar 2014 15:51:37 -0800 (PST)
\r
7 X-Virus-Scanned: Debian amavisd-new at olra.theworths.org
\r
11 X-Spam-Status: No, score=-0.7 tagged_above=-999 required=5
\r
12 tests=[RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled
\r
13 Received: from olra.theworths.org ([127.0.0.1])
\r
14 by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)
\r
15 with ESMTP id JQsxD92l1xwf for <notmuch@notmuchmail.org>;
\r
16 Sat, 8 Mar 2014 15:51:33 -0800 (PST)
\r
17 Received: from dmz-mailsec-scanner-5.mit.edu (dmz-mailsec-scanner-5.mit.edu
\r
19 by olra.theworths.org (Postfix) with ESMTP id 3EB01431FBD
\r
20 for <notmuch@notmuchmail.org>; Sat, 8 Mar 2014 15:51:33 -0800 (PST)
\r
21 X-AuditID: 12074422-f79526d000000c47-3b-531bad04f91f
\r
22 Received: from mailhub-auth-4.mit.edu ( [18.7.62.39])
\r
23 (using TLS with cipher AES256-SHA (256/256 bits))
\r
24 (Client did not present a certificate)
\r
25 by dmz-mailsec-scanner-5.mit.edu (Symantec Messaging Gateway) with SMTP
\r
26 id 89.81.03143.40DAB135; Sat, 8 Mar 2014 18:51:32 -0500 (EST)
\r
27 Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11])
\r
28 by mailhub-auth-4.mit.edu (8.13.8/8.9.2) with ESMTP id s28NpVfg015055;
\r
29 Sat, 8 Mar 2014 18:51:32 -0500
\r
30 Received: from awakening.csail.mit.edu (awakening.csail.mit.edu [18.26.4.91])
\r
31 (authenticated bits=0)
\r
32 (User authenticated as amdragon@ATHENA.MIT.EDU)
\r
33 by outgoing.mit.edu (8.13.8/8.12.4) with ESMTP id s28NpRoo013295
\r
34 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT);
\r
35 Sat, 8 Mar 2014 18:51:28 -0500
\r
36 Received: from amthrax by awakening.csail.mit.edu with local (Exim 4.80)
\r
37 (envelope-from <amdragon@mit.edu>)
\r
38 id 1WMR1b-0001qU-Ay; Sat, 08 Mar 2014 18:51:27 -0500
\r
39 Date: Sat, 8 Mar 2014 18:51:27 -0500
\r
40 From: Austin Clements <amdragon@MIT.EDU>
\r
41 To: jani@nikula.org, David Bremner <david@tethera.net>
\r
42 Subject: Re: [Patch v3 04/15] lib: make folder: prefix literal
\r
43 Message-ID: <20140308235127.GL4709@mit.edu>
\r
44 References: <1394313585-28422-1-git-send-email-david@tethera.net>
\r
45 <1394313585-28422-5-git-send-email-david@tethera.net>
\r
47 Content-Type: text/plain; charset=us-ascii
\r
48 Content-Disposition: inline
\r
49 In-Reply-To: <1394313585-28422-5-git-send-email-david@tethera.net>
\r
50 User-Agent: Mutt/1.5.21 (2010-09-15)
\r
51 X-Brightmail-Tracker:
\r
52 H4sIAAAAAAAAA+NgFupileLIzCtJLcpLzFFi42IRYrdT12VZKx1s0PhB1+JGazejRdN0Z4vr
\r
53 N2cyOzB73Lr/mt3j2apbzB5bDr1nDmCO4rJJSc3JLEst0rdL4Mr43/+CvWC7V8XXd2fZGhhv
\r
54 WHQxcnJICJhIHH42jw3CFpO4cG89kM3FISQwm0niw9Y/rBDOBkaJLRvWM4FUCQmcYpK428MO
\r
55 YS9hlJjwyw7EZhFQkVi3bhZYDZuAhsS2/csZuxg5OEQEzCSmLa8FCTMLSEt8+90MViIsYCfR
\r
56 9/IAC4jNK6AtMedQMxvEyEqJrYfOQMUFJU7OfMIC0aslcePfSyaQkSBzlv/jAAlzCjhJtO/o
\r
57 YASxRYEumHJyG9sERqFZSLpnIemehdC9gJF5FaNsSm6Vbm5iZk5xarJucXJiXl5qka6pXm5m
\r
58 iV5qSukmRnCQuyjtYPx5UOkQowAHoxIP74NJ0sFCrIllxZW5hxglOZiURHk7VwCF+JLyUyoz
\r
59 Eosz4otKc1KLDzFKcDArifC+7ATK8aYkVlalFuXDpKQ5WJTEefvOSgQLCaQnlqRmp6YWpBbB
\r
60 ZGU4OJQkeIPWADUKFqWmp1akZeaUIKSZODhBhvMADS8EqeEtLkjMLc5Mh8ifYlSUEuetWQ2U
\r
61 EABJZJTmwfXCktArRnGgV4R5a0DaeYAJDK77FdBgJqDBzcelQAaXJCKkpBoYVRZMWMjvadNd
\r
62 GXXb5o3Q2WBT3f64zrqok1+WWq31zT9uwaDP3Dgxc7ZlZNseoekr1hv/0Fz7fNWTd+1/di3Y
\r
63 37xAcINudJLuAp9nx75Kn3r2YK/crfqIJarL3BMuLuFKO8LFc+D563O3Jm0OXTW/LWum78uN
\r
64 OXG1z2bImTPNZG3YGnpCWte/VImlOCPRUIu5qDgRAOpf5bAdAwAA
\r
65 Cc: notmuch@notmuchmail.org
\r
66 X-BeenThere: notmuch@notmuchmail.org
\r
67 X-Mailman-Version: 2.1.13
\r
69 List-Id: "Use and development of the notmuch mail system."
\r
70 <notmuch.notmuchmail.org>
\r
71 List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,
\r
72 <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>
\r
73 List-Archive: <http://notmuchmail.org/pipermail/notmuch>
\r
74 List-Post: <mailto:notmuch@notmuchmail.org>
\r
75 List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>
\r
76 List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,
\r
77 <mailto:notmuch-request@notmuchmail.org?subject=subscribe>
\r
78 X-List-Received-Date: Sat, 08 Mar 2014 23:51:37 -0000
\r
80 Quoth David Bremner on Mar 08 at 5:19 pm:
\r
81 > From: Jani Nikula <jani@nikula.org>
\r
83 > In xapian terms, convert folder: prefix from probabilistic to boolean
\r
84 > prefix, matching the paths, relative form the maildir root, of the
\r
88 > message files, ignoring the maildir new and cur leaf directories.
\r
90 > folder:foo matches all message files in foo, foo/new, and foo/cur.
\r
92 > folder:foo/new does *not* match message files in foo/new.
\r
94 > folder:"" matches all message files in the top level maildir and its
\r
95 > new and cur subdirectories.
\r
97 > This change constitutes a database change: bump the database version
\r
98 > and add database upgrade support for folder: terms. The upgrade also
\r
101 > lib/database.cc | 38 ++++++++++++++++++++++--
\r
102 > lib/message.cc | 80 ++++++++++++++++++++++++++++++++++++++++++++-------
\r
103 > lib/notmuch-private.h | 3 ++
\r
104 > 3 files changed, 108 insertions(+), 13 deletions(-)
\r
106 > diff --git a/lib/database.cc b/lib/database.cc
\r
107 > index 93cc7f5..186e3a7 100644
\r
108 > --- a/lib/database.cc
\r
109 > +++ b/lib/database.cc
\r
110 > @@ -42,7 +42,7 @@ typedef struct {
\r
111 > const char *prefix;
\r
114 > -#define NOTMUCH_DATABASE_VERSION 1
\r
115 > +#define NOTMUCH_DATABASE_VERSION 2
\r
117 > #define STRINGIFY(s) _SUB_STRINGIFY(s)
\r
118 > #define _SUB_STRINGIFY(s) #s
\r
119 > @@ -210,6 +210,7 @@ static prefix_t BOOLEAN_PREFIX_EXTERNAL[] = {
\r
123 > + { "folder", "XFOLDER:" },
\r
125 It took me a while to figure out that the ":" here means that Xapian
\r
126 will unconditionally use a ":" after the prefix, instead of only using
\r
127 a ":" when the first letter following the prefix is upper-case ASCII.
\r
128 Maybe I was only confused by this because I simultaneously knew too
\r
129 much and not enough about Xapian, but it might be worth a comment.
\r
132 /* Without the ":", since this is a multi-letter prefix, Xapian
\r
133 * will add a colon itself if the first letter of the path is
\r
134 * upper-case ASCII. Including the ":" forces there to always be
\r
135 * a colon, which keeps our own logic simpler. */
\r
139 > static prefix_t PROBABILISTIC_PREFIX[]= {
\r
140 > @@ -217,7 +218,6 @@ static prefix_t PROBABILISTIC_PREFIX[]= {
\r
142 > { "attachment", "XATTACHMENT" },
\r
143 > { "subject", "XSUBJECT"},
\r
144 > - { "folder", "XFOLDER"}
\r
148 > @@ -1168,6 +1168,40 @@ notmuch_database_upgrade (notmuch_database_t *notmuch,
\r
153 > + * Prior to version 2, the "folder:" prefix was probabilistic and
\r
154 > + * stemmed. Change it to the current boolean prefix. Add "path:"
\r
155 > + * prefixes while at it.
\r
157 > + if (version < 2) {
\r
158 > + notmuch_query_t *query = notmuch_query_create (notmuch, "");
\r
160 Three space indentation and no tabs? (It looks like this was in
\r
161 Jani's v2, also. I'm guessing at some point there was a copy-paste
\r
162 from a diff with tabs converted to spaces?)
\r
164 > + notmuch_messages_t *messages;
\r
165 > + notmuch_message_t *message;
\r
168 > + total = notmuch_query_count_messages (query);
\r
170 > + for (messages = notmuch_query_search_messages (query);
\r
171 > + notmuch_messages_valid (messages);
\r
172 > + notmuch_messages_move_to_next (messages)) {
\r
173 > + if (do_progress_notify) {
\r
174 > + progress_notify (closure, (double) count / total);
\r
175 > + do_progress_notify = 0;
\r
178 > + message = notmuch_messages_get (messages);
\r
180 > + _notmuch_message_upgrade_folder (message);
\r
181 > + _notmuch_message_sync (message);
\r
183 > + notmuch_message_destroy (message);
\r
188 > + notmuch_query_destroy (query);
\r
191 > db->set_metadata ("version", STRINGIFY (NOTMUCH_DATABASE_VERSION));
\r
194 > diff --git a/lib/message.cc b/lib/message.cc
\r
195 > index 21abe8e..31cb9f1 100644
\r
196 > --- a/lib/message.cc
\r
197 > +++ b/lib/message.cc
\r
198 > @@ -504,6 +504,56 @@ _notmuch_message_remove_terms (notmuch_message_t *message, const char *prefix)
\r
202 > +/* Return true if p points at "new" or "cur". */
\r
203 > +static bool is_maildir (const char *p)
\r
205 > + return strcmp (p, "cur") == 0 || strcmp (p, "new") == 0;
\r
208 > +/* Add "folder:" term for directory. */
\r
209 > +static notmuch_status_t
\r
210 > +_notmuch_message_add_folder_terms (notmuch_message_t *message,
\r
211 > + const char *directory)
\r
213 > + char *folder, *last;
\r
215 > + folder = talloc_strdup (NULL, directory);
\r
217 > + return NOTMUCH_STATUS_OUT_OF_MEMORY;
\r
219 Same formatting problem in this chunk.
\r
223 > + * If the message file is in a leaf directory named "new" or
\r
224 > + * "cur", presume maildir and index the parent directory. Thus a
\r
225 > + * "folder:" prefix search matches messages in the specified
\r
226 > + * maildir folder, i.e. in the specified directory and its "new"
\r
227 > + * and "cur" subdirectories.
\r
229 > + * Note that this means the "folder:" prefix can't be used for
\r
230 > + * distinguishing between message files in "new" or "cur". The
\r
231 > + * "path:" prefix needs to be used for that.
\r
233 > + * Note the deliberate difference to _filename_is_in_maildir(). We
\r
234 > + * don't want to index different things depending on the existence
\r
235 > + * or non-existence of all maildir sibling directories "new",
\r
236 > + * "cur", and "tmp". Doing so would be surprising, and difficult
\r
237 > + * for the user to fix in case all subdirectories were not in
\r
238 > + * place during indexing.
\r
240 > + last = strrchr (folder, '/');
\r
242 > + if (is_maildir (last + 1))
\r
244 > + } else if (is_maildir (folder)) {
\r
245 > + *folder = '\0';
\r
248 > + _notmuch_message_add_term (message, "folder", folder);
\r
250 > + talloc_free (folder);
\r
252 > + return NOTMUCH_STATUS_SUCCESS;
\r
255 > #define RECURSIVE_SUFFIX "/**"
\r
257 > /* Add "path:" terms for directory. */
\r
258 > @@ -570,9 +620,8 @@ _notmuch_message_add_directory_terms (void *ctx, notmuch_message_t *message)
\r
259 > directory = _notmuch_database_get_directory_path (ctx,
\r
260 > message->notmuch,
\r
262 > - if (strlen (directory))
\r
263 > - _notmuch_message_gen_terms (message, "folder", directory);
\r
265 > + _notmuch_message_add_folder_terms (message, directory);
\r
266 > _notmuch_message_add_path_terms (message, directory);
\r
269 > @@ -610,9 +659,7 @@ _notmuch_message_add_filename (notmuch_message_t *message,
\r
270 > * notmuch_directory_get_child_files() . */
\r
271 > _notmuch_message_add_term (message, "file-direntry", direntry);
\r
273 > - /* New terms allow user to search with folder: specification. */
\r
274 > - _notmuch_message_gen_terms (message, "folder", directory);
\r
276 > + _notmuch_message_add_folder_terms (message, directory);
\r
277 > _notmuch_message_add_path_terms (message, directory);
\r
279 > talloc_free (local);
\r
280 > @@ -637,8 +684,6 @@ _notmuch_message_remove_filename (notmuch_message_t *message,
\r
281 > const char *filename)
\r
283 > void *local = talloc_new (message);
\r
284 > - const char *folder_prefix = _find_prefix ("folder");
\r
285 > - char *zfolder_prefix = talloc_asprintf(local, "Z%s", folder_prefix);
\r
287 > notmuch_private_status_t private_status;
\r
288 > notmuch_status_t status;
\r
289 > @@ -659,10 +704,7 @@ _notmuch_message_remove_filename (notmuch_message_t *message,
\r
290 > /* Re-synchronize "folder:" and "path:" terms for this message. */
\r
292 > /* Remove all "folder:" terms. */
\r
293 > - _notmuch_message_remove_terms (message, folder_prefix);
\r
295 > - /* Remove all "folder:" stemmed terms. */
\r
296 > - _notmuch_message_remove_terms (message, zfolder_prefix);
\r
297 > + _notmuch_message_remove_terms (message, _find_prefix ("folder"));
\r
299 > /* Remove all "path:" terms. */
\r
300 > _notmuch_message_remove_terms (message, _find_prefix ("path"));
\r
301 > @@ -675,6 +717,22 @@ _notmuch_message_remove_filename (notmuch_message_t *message,
\r
305 > +/* Upgrade the "folder:" prefix from V1 to V2. */
\r
306 > +#define FOLDER_PREFIX_V1 "XFOLDER"
\r
307 > +#define ZFOLDER_PREFIX_V1 "Z" FOLDER_PREFIX_V1
\r
309 > +_notmuch_message_upgrade_folder (notmuch_message_t *message)
\r
311 > + /* Remove all old "folder:" terms. */
\r
312 > + _notmuch_message_remove_terms (message, FOLDER_PREFIX_V1);
\r
314 > + /* Remove all old "folder:" stemmed terms. */
\r
315 > + _notmuch_message_remove_terms (message, ZFOLDER_PREFIX_V1);
\r
317 > + /* Add new boolean "folder:" and "path:" terms. */
\r
318 > + _notmuch_message_add_directory_terms (message, message);
\r
322 > _notmuch_message_talloc_copy_data (notmuch_message_t *message)
\r
324 > diff --git a/lib/notmuch-private.h b/lib/notmuch-private.h
\r
325 > index af185c7..59eb2bc 100644
\r
326 > --- a/lib/notmuch-private.h
\r
327 > +++ b/lib/notmuch-private.h
\r
328 > @@ -263,6 +263,9 @@ _notmuch_message_gen_terms (notmuch_message_t *message,
\r
330 > _notmuch_message_upgrade_filename_storage (notmuch_message_t *message);
\r
333 > +_notmuch_message_upgrade_folder (notmuch_message_t *message);
\r
336 > _notmuch_message_add_filename (notmuch_message_t *message,
\r
337 > const char *filename);
\r