1 Return-Path: <sojkam1@fel.cvut.cz>
\r
2 X-Original-To: notmuch@notmuchmail.org
\r
3 Delivered-To: notmuch@notmuchmail.org
\r
4 Received: from localhost (localhost [127.0.0.1])
\r
5 by olra.theworths.org (Postfix) with ESMTP id D0B4B40CAE9
\r
6 for <notmuch@notmuchmail.org>; Tue, 11 May 2010 05:15:01 -0700 (PDT)
\r
7 X-Virus-Scanned: Debian amavisd-new at olra.theworths.org
\r
11 X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5
\r
12 tests=[BAYES_00=-1.9] autolearn=ham
\r
13 Received: from olra.theworths.org ([127.0.0.1])
\r
14 by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)
\r
15 with ESMTP id 6RKDn6TjOyXo for <notmuch@notmuchmail.org>;
\r
16 Tue, 11 May 2010 05:14:47 -0700 (PDT)
\r
17 Received: from max.feld.cvut.cz (max.feld.cvut.cz [147.32.192.36])
\r
18 by olra.theworths.org (Postfix) with ESMTP id 87D2B4196F4
\r
19 for <notmuch@notmuchmail.org>; Tue, 11 May 2010 05:14:34 -0700 (PDT)
\r
20 Received: from localhost (unknown [192.168.200.4])
\r
21 by max.feld.cvut.cz (Postfix) with ESMTP id 8465819F33F4;
\r
22 Tue, 11 May 2010 14:14:33 +0200 (CEST)
\r
23 X-Virus-Scanned: IMAP AMAVIS
\r
24 Received: from max.feld.cvut.cz ([192.168.200.1])
\r
25 by localhost (styx.feld.cvut.cz [192.168.200.4]) (amavisd-new,
\r
27 with ESMTP id mIligpFZ4x-E; Tue, 11 May 2010 14:14:31 +0200 (CEST)
\r
28 Received: from imap.feld.cvut.cz (imap.feld.cvut.cz [147.32.192.34])
\r
29 by max.feld.cvut.cz (Postfix) with ESMTP id 84A8119F33F8;
\r
30 Tue, 11 May 2010 14:14:31 +0200 (CEST)
\r
31 Received: from steelpick.2x.cz (k335-30.felk.cvut.cz [147.32.86.30])
\r
32 (Authenticated sender: sojkam1)
\r
33 by imap.feld.cvut.cz (Postfix) with ESMTPSA id 7522BFA003;
\r
34 Tue, 11 May 2010 14:14:31 +0200 (CEST)
\r
35 Received: from wsh by steelpick.2x.cz with local (Exim 4.71)
\r
36 (envelope-from <sojkam1@fel.cvut.cz>)
\r
37 id 1OBoM3-0005st-6u; Tue, 11 May 2010 14:14:31 +0200
\r
38 From: Michal Sojka <sojkam1@fel.cvut.cz>
\r
39 To: notmuch@notmuchmail.org
\r
40 Subject: [PATCH 2/4] Maildir synchronization
\r
41 Date: Tue, 11 May 2010 14:14:19 +0200
\r
42 Message-Id: <1273580061-22580-3-git-send-email-sojkam1@fel.cvut.cz>
\r
43 X-Mailer: git-send-email 1.7.1
\r
44 In-Reply-To: <1273580061-22580-1-git-send-email-sojkam1@fel.cvut.cz>
\r
45 References: <1273580061-22580-1-git-send-email-sojkam1@fel.cvut.cz>
\r
46 X-BeenThere: notmuch@notmuchmail.org
\r
47 X-Mailman-Version: 2.1.13
\r
49 List-Id: "Use and development of the notmuch mail system."
\r
50 <notmuch.notmuchmail.org>
\r
51 List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,
\r
52 <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>
\r
53 List-Archive: <http://notmuchmail.org/pipermail/notmuch>
\r
54 List-Post: <mailto:notmuch@notmuchmail.org>
\r
55 List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>
\r
56 List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,
\r
57 <mailto:notmuch-request@notmuchmail.org?subject=subscribe>
\r
58 X-List-Received-Date: Tue, 11 May 2010 12:15:02 -0000
\r
60 This patch allows bi-directional synchronization between maildir
\r
61 flags and certain tags. The flag-to-tag mapping is defined by flag2tag
\r
64 The synchronization works this way:
\r
66 1) Whenever notmuch new is executed, the following happens:
\r
67 o New messages are tagged with configured new_tags.
\r
68 o For new or renamed messages with maildir info present in the file
\r
69 name, the tags defined in flag2tag are either added or removed
\r
70 depending on the flags from the file name.
\r
72 2) Whenever notmuch tag (or notmuch restore) is executed, a new set of
\r
73 flags based on the tags is constructed for every message and a new
\r
74 file name is prepared based on the old file name but with the new
\r
75 flags. If the flags differs and the old message was in 'new'
\r
76 directory then this is replaced with 'cur' in the new file name. If
\r
77 the new and old file names differ, the file is renamed and notmuch
\r
78 database is updated accordingly.
\r
80 The rename happens before the database is updated. In case of crash
\r
81 between rename and database update, the next run of notmuch new
\r
82 brings the database in sync with the mail store again.
\r
84 There is currenlty one known issue: Viewing/storing of attachments of
\r
85 unread messages doesn't work. The reason is that when you view the
\r
86 message its unread tag is removed which leads to rename of the file,
\r
87 but Emacs still uses the original name to access the attachment.
\r
89 Workaround: close the message and open it again.
\r
91 lib/database.cc | 7 ++
\r
92 lib/message.cc | 226 +++++++++++++++++++++++++++++++++++++++++++++++++
\r
93 lib/notmuch-private.h | 4 +
\r
94 lib/notmuch.h | 7 ++
\r
95 notmuch-new.c | 3 +-
\r
96 5 files changed, 246 insertions(+), 1 deletions(-)
\r
98 diff --git a/lib/database.cc b/lib/database.cc
\r
99 index 694b7ec..908bbaa 100644
\r
100 --- a/lib/database.cc
\r
101 +++ b/lib/database.cc
\r
102 @@ -1585,6 +1585,13 @@ notmuch_database_add_message (notmuch_database_t *notmuch,
\r
104 _notmuch_message_add_filename (message, filename);
\r
106 + /* This is a new message or it has a new filename and as such,
\r
107 + * its tags in database either do not exists or might be out
\r
108 + * of date. We assign the tags later in notmuch new, but until
\r
109 + * then we should not synchronize the tags back to the maildir
\r
110 + * flags (if notmuch is configured to do so). */
\r
111 + notmuch_message_set_flag(message, NOTMUCH_MESSAGE_FLAG_TAGS_INVALID, TRUE);
\r
113 /* Is this a newly created message object? */
\r
114 if (private_status == NOTMUCH_PRIVATE_STATUS_NO_DOCUMENT_FOUND) {
\r
115 _notmuch_message_add_term (message, "type", "mail");
\r
116 diff --git a/lib/message.cc b/lib/message.cc
\r
117 index 4b2f98f..0de3f01 100644
\r
118 --- a/lib/message.cc
\r
119 +++ b/lib/message.cc
\r
120 @@ -43,6 +43,24 @@ struct _notmuch_message {
\r
121 Xapian::Document doc;
\r
124 +#define ARRAY_SIZE(arr) (sizeof (arr) / sizeof (arr[0]))
\r
126 +struct maildir_flag_tag {
\r
132 +/* ASCII ordered table of Maildir flags and associated tags */
\r
133 +struct maildir_flag_tag flag2tag[] = {
\r
134 + { 'D', "draft", false},
\r
135 + { 'F', "flagged", false},
\r
136 + { 'P', "passed", false},
\r
137 + { 'R', "replied", false},
\r
138 + { 'S', "unread", true },
\r
139 + { 'T', "delete", false},
\r
142 /* We end up having to call the destructor explicitly because we had
\r
143 * to use "placement new" in order to initialize C++ objects within a
\r
144 * block that we allocated with talloc. So C++ is making talloc
\r
145 @@ -568,15 +586,29 @@ _notmuch_message_set_date (notmuch_message_t *message,
\r
146 Xapian::sortable_serialise (time_value));
\r
149 +static notmuch_private_status_t
\r
150 +_notmuch_message_tags_to_maildir (notmuch_message_t *message);
\r
152 /* Synchronize changes made to message->doc out into the database. */
\r
154 _notmuch_message_sync (notmuch_message_t *message)
\r
156 Xapian::WritableDatabase *db;
\r
157 + notmuch_private_status_t status;
\r
159 if (message->notmuch->mode == NOTMUCH_DATABASE_MODE_READ_ONLY)
\r
162 + if (// todo_sync_enabled &&
\r
163 + !notmuch_message_get_flag(message, NOTMUCH_MESSAGE_FLAG_TAGS_INVALID)) {
\r
164 + status = _notmuch_message_tags_to_maildir (message);
\r
165 + if (status != NOTMUCH_PRIVATE_STATUS_SUCCESS) {
\r
166 + fprintf (stderr, "Error: Cannot sync tags to maildir (%s)\n",
\r
167 + notmuch_status_to_string ((notmuch_status_t)status));
\r
168 + /* Exit to avoid unsynchronized mailstore. */
\r
172 db = static_cast <Xapian::WritableDatabase *> (message->notmuch->xapian_db);
\r
173 db->replace_document (message->doc_id, message->doc);
\r
175 @@ -688,6 +720,44 @@ _notmuch_message_remove_term (notmuch_message_t *message,
\r
176 return NOTMUCH_PRIVATE_STATUS_SUCCESS;
\r
179 +/* Change the message filename stored in the database.
\r
181 + * This change will not be reflected in the database until the next
\r
182 + * call to _notmuch_message_sync.
\r
184 +notmuch_private_status_t
\r
185 +_notmuch_message_rename (notmuch_message_t *message,
\r
186 + const char *new_filename)
\r
188 + void *local = talloc_new (message);
\r
190 + Xapian::PostingIterator i, end;
\r
191 + Xapian::Document document;
\r
192 + notmuch_private_status_t pstatus;
\r
193 + notmuch_status_t status;
\r
194 + const char *old_filename;
\r
196 + old_filename = notmuch_message_get_filename(message);
\r
197 + old_filename = talloc_reference(local, old_filename);
\r
198 + if (unlikely(!old_filename))
\r
199 + return NOTMUCH_PRIVATE_STATUS_OUT_OF_MEMORY;
\r
201 + status = _notmuch_message_add_filename (message, new_filename);
\r
203 + return (notmuch_private_status_t)status;
\r
205 + status = _notmuch_database_filename_to_direntry (local, message->notmuch,
\r
206 + old_filename, &direntry);
\r
208 + return (notmuch_private_status_t)status;
\r
210 + pstatus = _notmuch_message_remove_term (message, "file-direntry", direntry);
\r
212 + talloc_free (local);
\r
218 notmuch_message_add_tag (notmuch_message_t *message, const char *tag)
\r
220 @@ -745,6 +815,162 @@ notmuch_message_remove_tag (notmuch_message_t *message, const char *tag)
\r
224 +notmuch_message_maildir_to_tags (notmuch_message_t *message, const char *filename)
\r
226 + const char *flags, *p;
\r
228 + bool valid, unread;
\r
230 + notmuch_status_t status;
\r
232 + flags = strstr (filename, ":2,");
\r
234 + return NOTMUCH_STATUS_FILE_NOT_EMAIL;
\r
237 + /* Check that the letters are valid Maildir flags */
\r
240 + for (p=flags; valid && *p; p++) {
\r
248 + if (*p > f) f=*p;
\r
249 + else valid = false;
\r
256 + fprintf (stderr, "Warning: Invalid maildir flags in filename %s\n", filename);
\r
257 + return NOTMUCH_STATUS_FILE_NOT_EMAIL;
\r
260 + status = notmuch_message_freeze (message);
\r
264 + for (i = 0; i < ARRAY_SIZE(flag2tag); i++) {
\r
265 + if ((strchr (flags, flag2tag[i].flag) != NULL) ^ flag2tag[i].inverse) {
\r
266 + status = notmuch_message_add_tag (message, flag2tag[i].tag);
\r
268 + status = notmuch_message_remove_tag (message, flag2tag[i].tag);
\r
273 + status = notmuch_message_thaw (message);
\r
275 + /* From now on, we can synchronize the tags from the database to
\r
276 + * the mailstore. */
\r
277 + notmuch_message_set_flag (message, NOTMUCH_MESSAGE_FLAG_TAGS_INVALID, FALSE);
\r
282 +maildir_get_new_flags(notmuch_message_t *message, char *flags)
\r
284 + notmuch_tags_t *tags;
\r
289 + for (i = 0; i < ARRAY_SIZE(flag2tag); i++)
\r
290 + flags[i] = flag2tag[i].inverse ? flag2tag[i].flag : '\0';
\r
292 + for (tags = notmuch_message_get_tags (message);
\r
293 + notmuch_tags_valid (tags);
\r
294 + notmuch_tags_move_to_next (tags))
\r
296 + tag = notmuch_tags_get (tags);
\r
297 + for (i = 0; i < ARRAY_SIZE(flag2tag); i++) {
\r
298 + if (strcmp(tag, flag2tag[i].tag) == 0)
\r
299 + flags[i] = flag2tag[i].inverse ? '\0' : flag2tag[i].flag;
\r
304 + for (i = 0; i < ARRAY_SIZE(flag2tag); i++) {
\r
312 +maildir_get_subdir (char *filename)
\r
314 + char *p, *subdir = NULL;
\r
316 + p = filename + strlen (filename) - 1;
\r
317 + while (p > filename + 3 && *p != '/')
\r
321 + if (subdir > filename && *(subdir - 1) != '/')
\r
327 +/* Rename the message file so that maildir flags corresponds to the
\r
328 + * tags and, if aplicable, move the message from new/ to cur/. */
\r
329 +static notmuch_private_status_t
\r
330 +_notmuch_message_tags_to_maildir (notmuch_message_t *message)
\r
332 + char flags[ARRAY_SIZE(flag2tag)+1];
\r
333 + const char *filename, *p;
\r
334 + char *filename_new, *subdir = NULL;
\r
337 + maildir_get_new_flags (message, flags);
\r
339 + filename = notmuch_message_get_filename (message);
\r
340 + /* TODO: Iterate over all file names. */
\r
341 + p = strstr(filename, ":2,");
\r
342 + if ((p && strcmp (p+3, flags) == 0) ||
\r
343 + (!p && flags[0] == '\0')) {
\r
344 + // Return if flags are not to be changed - this suppresses
\r
345 + // moving the message from new/ to cur/ during initial
\r
347 + return NOTMUCH_PRIVATE_STATUS_SUCCESS;
\r
350 + p = filename + strlen(filename);
\r
352 + filename_new = (char*)talloc_size(message, (p-filename) + 3 + sizeof(flags));
\r
353 + if (unlikely (filename_new == NULL))
\r
354 + return NOTMUCH_PRIVATE_STATUS_OUT_OF_MEMORY;
\r
355 + memcpy(filename_new, filename, p-filename);
\r
356 + filename_new[p-filename] = '\0';
\r
358 + /* If message is in new/ move it under cur/. */
\r
359 + subdir = maildir_get_subdir (filename_new);
\r
360 + if (subdir && memcmp (subdir, "new/", 4) == 0)
\r
361 + memcpy (subdir, "cur/", 4);
\r
363 + strcpy (filename_new+(p-filename), ":2,");
\r
364 + strcpy (filename_new+(p-filename)+3, flags);
\r
366 + if (strcmp (filename, filename_new) != 0) {
\r
367 + ret = rename (filename, filename_new);
\r
369 + perror (talloc_asprintf (message, "rename of %s to %s failed",
\r
370 + filename, filename_new));
\r
373 + return _notmuch_message_rename (message, filename_new);
\r
374 + /* _notmuch_message_sync is our caller. Do not call it here. */
\r
376 + return NOTMUCH_PRIVATE_STATUS_SUCCESS;
\r
380 notmuch_message_remove_all_tags (notmuch_message_t *message)
\r
382 notmuch_private_status_t private_status;
\r
383 diff --git a/lib/notmuch-private.h b/lib/notmuch-private.h
\r
384 index 3768d6f..ba11498 100644
\r
385 --- a/lib/notmuch-private.h
\r
386 +++ b/lib/notmuch-private.h
\r
387 @@ -246,6 +246,10 @@ notmuch_status_t
\r
388 _notmuch_message_add_filename (notmuch_message_t *message,
\r
389 const char *filename);
\r
391 +notmuch_private_status_t
\r
392 +_notmuch_message_rename (notmuch_message_t *message,
\r
393 + const char *new_filename);
\r
396 _notmuch_message_ensure_thread_id (notmuch_message_t *message);
\r
398 diff --git a/lib/notmuch.h b/lib/notmuch.h
\r
399 index 0ba1416..408d633 100644
\r
400 --- a/lib/notmuch.h
\r
401 +++ b/lib/notmuch.h
\r
402 @@ -770,6 +770,7 @@ notmuch_message_get_filename (notmuch_message_t *message);
\r
403 /* Message flags */
\r
404 typedef enum _notmuch_message_flag {
\r
405 NOTMUCH_MESSAGE_FLAG_MATCH,
\r
406 + NOTMUCH_MESSAGE_FLAG_TAGS_INVALID,
\r
407 } notmuch_message_flag_t;
\r
409 /* Get a value of a flag for the email corresponding to 'message'. */
\r
410 @@ -886,6 +887,12 @@ notmuch_message_remove_tag (notmuch_message_t *message, const char *tag);
\r
412 notmuch_message_remove_all_tags (notmuch_message_t *message);
\r
414 +/* Add or remove tags based on the maildir flags in the file name.
\r
417 +notmuch_message_maildir_to_tags (notmuch_message_t *message,
\r
418 + const char *filename);
\r
420 /* Freeze the current state of 'message' within the database.
\r
422 * This means that changes to the message state, (via
\r
423 diff --git a/notmuch-new.c b/notmuch-new.c
\r
424 index 8818728..ed3f944 100644
\r
425 --- a/notmuch-new.c
\r
426 +++ b/notmuch-new.c
\r
427 @@ -410,10 +410,11 @@ add_files_recursive (notmuch_database_t *notmuch,
\r
428 state->added_messages++;
\r
429 for (tag=state->new_tags; *tag != NULL; tag++)
\r
430 notmuch_message_add_tag (message, *tag);
\r
431 + notmuch_message_maildir_to_tags (message, next);
\r
433 /* Non-fatal issues (go on to next file) */
\r
434 case NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID:
\r
435 - /* Stay silent on this one. */
\r
436 + notmuch_message_maildir_to_tags (message, next);
\r
438 case NOTMUCH_STATUS_FILE_NOT_EMAIL:
\r
439 fprintf (stderr, "Note: Ignoring non-mail file: %s\n",
\r