1 Return-Path: <m.walters@qmul.ac.uk>
\r
2 X-Original-To: notmuch@notmuchmail.org
\r
3 Delivered-To: notmuch@notmuchmail.org
\r
4 Received: from localhost (localhost [127.0.0.1])
\r
5 by olra.theworths.org (Postfix) with ESMTP id 4D882431FBD
\r
6 for <notmuch@notmuchmail.org>; Mon, 24 Jun 2013 12:02:59 -0700 (PDT)
\r
7 X-Virus-Scanned: Debian amavisd-new at olra.theworths.org
\r
11 X-Spam-Status: No, score=-1.098 tagged_above=-999 required=5
\r
12 tests=[DKIM_ADSP_CUSTOM_MED=0.001, FREEMAIL_FROM=0.001,
\r
13 NML_ADSP_CUSTOM_MED=1.2, RCVD_IN_DNSWL_MED=-2.3] autolearn=disabled
\r
14 Received: from olra.theworths.org ([127.0.0.1])
\r
15 by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)
\r
16 with ESMTP id wWEnA8xp7bVy for <notmuch@notmuchmail.org>;
\r
17 Mon, 24 Jun 2013 12:02:53 -0700 (PDT)
\r
18 Received: from mail2.qmul.ac.uk (mail2.qmul.ac.uk [138.37.6.6])
\r
19 (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
\r
20 (No client certificate requested)
\r
21 by olra.theworths.org (Postfix) with ESMTPS id E4A57431FAF
\r
22 for <notmuch@notmuchmail.org>; Mon, 24 Jun 2013 12:02:52 -0700 (PDT)
\r
23 Received: from smtp.qmul.ac.uk ([138.37.6.40])
\r
24 by mail2.qmul.ac.uk with esmtp (Exim 4.71)
\r
25 (envelope-from <m.walters@qmul.ac.uk>)
\r
26 id 1UrC2L-00040F-HK; Mon, 24 Jun 2013 20:02:50 +0100
\r
27 Received: from 93-97-24-31.zone5.bethere.co.uk ([93.97.24.31] helo=localhost)
\r
28 by smtp.qmul.ac.uk with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.71)
\r
29 (envelope-from <m.walters@qmul.ac.uk>)
\r
30 id 1UrC2K-0000ro-PX; Mon, 24 Jun 2013 20:02:49 +0100
\r
31 From: Mark Walters <markwalters1009@gmail.com>
\r
32 To: Peter Wang <novalazy@gmail.com>, notmuch@notmuchmail.org
\r
33 Subject: Re: [PATCH v7b] cli: add insert command
\r
34 In-Reply-To: <1371990045-20890-1-git-send-email-novalazy@gmail.com>
\r
35 References: <20130623221942.GA18938@hili.localdomain>
\r
36 <1371990045-20890-1-git-send-email-novalazy@gmail.com>
\r
37 User-Agent: Notmuch/0.15.2+171~ge2f30a2 (http://notmuchmail.org) Emacs/23.4.1
\r
38 (x86_64-pc-linux-gnu)
\r
39 Date: Mon, 24 Jun 2013 20:02:45 +0100
\r
40 Message-ID: <87wqpjtjqi.fsf@qmul.ac.uk>
\r
42 Content-Type: text/plain; charset=utf-8
\r
43 Content-Transfer-Encoding: quoted-printable
\r
44 X-Sender-Host-Address: 93.97.24.31
\r
45 X-QM-SPAM-Info: Sender has good ham record. :)
\r
46 X-QM-Body-MD5: 407a8c53009caaf73c30eb82d45c5e6b (of first 20000 bytes)
\r
47 X-SpamAssassin-Score: -0.0
\r
48 X-SpamAssassin-SpamBar: /
\r
49 X-SpamAssassin-Report: The QM spam filters have analysed this message to
\r
51 spam. We require at least 5.0 points to mark a message as spam.
\r
52 This message scored -0.0 points.
\r
53 Summary of the scoring:
\r
54 * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail
\r
55 provider * (markwalters1009[at]gmail.com)
\r
56 * -0.0 AWL AWL: From: address is in the auto white-list
\r
57 X-QM-Scan-Virus: ClamAV says the message is clean
\r
58 X-BeenThere: notmuch@notmuchmail.org
\r
59 X-Mailman-Version: 2.1.13
\r
61 List-Id: "Use and development of the notmuch mail system."
\r
62 <notmuch.notmuchmail.org>
\r
63 List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,
\r
64 <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>
\r
65 List-Archive: <http://notmuchmail.org/pipermail/notmuch>
\r
66 List-Post: <mailto:notmuch@notmuchmail.org>
\r
67 List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>
\r
68 List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,
\r
69 <mailto:notmuch-request@notmuchmail.org?subject=subscribe>
\r
70 X-List-Received-Date: Mon, 24 Jun 2013 19:02:59 -0000
\r
73 Just to confirm my +1 for this series with patch 7b.
\r
77 On Sun, 23 Jun 2013, Peter Wang <novalazy@gmail.com> wrote:
\r
78 > The notmuch insert command reads a message from standard input,
\r
79 > writes it to a Maildir folder, and then incorporates the message into
\r
80 > the notmuch database. Essentially it moves the functionality of
\r
81 > notmuch-deliver into notmuch.
\r
83 > Though it could be used as an alternative to notmuch new, the reason
\r
84 > I want this is to allow my notmuch frontend to add postponed or sent
\r
85 > messages to the mail store and notmuch database, without resorting to
\r
86 > another tool (e.g. notmuch-deliver) nor directly modifying the maildir.
\r
88 > Makefile.local | 1 +
\r
89 > notmuch-client.h | 3 +
\r
90 > notmuch-insert.c | 337 +++++++++++++++++++++++++++++++++++++++++++++++++=
\r
93 > 4 files changed, 343 insertions(+)
\r
94 > create mode 100644 notmuch-insert.c
\r
96 > diff --git a/Makefile.local b/Makefile.local
\r
97 > index 644623f..84043fe 100644
\r
98 > --- a/Makefile.local
\r
99 > +++ b/Makefile.local
\r
100 > @@ -261,6 +261,7 @@ notmuch_client_srcs =3D \
\r
101 > notmuch-config.c \
\r
102 > notmuch-count.c \
\r
104 > + notmuch-insert.c \
\r
106 > notmuch-reply.c \
\r
107 > notmuch-restore.c \
\r
108 > diff --git a/notmuch-client.h b/notmuch-client.h
\r
109 > index 4a3c7ac..dfe81e6 100644
\r
110 > --- a/notmuch-client.h
\r
111 > +++ b/notmuch-client.h
\r
112 > @@ -181,6 +181,9 @@ int
\r
113 > notmuch_new_command (notmuch_config_t *config, int argc, char *argv[]);
\r
116 > +notmuch_insert_command (notmuch_config_t *config, int argc, char *argv[]=
\r
120 > notmuch_reply_command (notmuch_config_t *config, int argc, char *argv[]);
\r
123 > diff --git a/notmuch-insert.c b/notmuch-insert.c
\r
124 > new file mode 100644
\r
125 > index 0000000..1228afa
\r
127 > +++ b/notmuch-insert.c
\r
128 > @@ -0,0 +1,337 @@
\r
129 > +/* notmuch - Not much of an email program, (just index and search)
\r
131 > + * Copyright =C2=A9 2013 Peter Wang
\r
133 > + * Based in part on notmuch-deliver
\r
134 > + * Copyright =C2=A9 2010 Ali Polatel
\r
136 > + * This program is free software: you can redistribute it and/or modify
\r
137 > + * it under the terms of the GNU General Public License as published by
\r
138 > + * the Free Software Foundation, either version 3 of the License, or
\r
139 > + * (at your option) any later version.
\r
141 > + * This program is distributed in the hope that it will be useful,
\r
142 > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
\r
143 > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
\r
144 > + * GNU General Public License for more details.
\r
146 > + * You should have received a copy of the GNU General Public License
\r
147 > + * along with this program. If not, see http://www.gnu.org/licenses/ .
\r
149 > + * Author: Peter Wang <novalazy@gmail.com>
\r
152 > +#include "notmuch-client.h"
\r
153 > +#include "tag-util.h"
\r
155 > +#include <sys/types.h>
\r
156 > +#include <sys/stat.h>
\r
157 > +#include <fcntl.h>
\r
159 > +static volatile sig_atomic_t interrupted;
\r
162 > +handle_sigint (unused (int sig))
\r
164 > + static char msg[] =3D "Stopping... \n";
\r
166 > + /* This write is "opportunistic", so it's okay to ignore the
\r
167 > + * result. It is not required for correctness, and if it does
\r
168 > + * fail or produce a short write, we want to get out of the signal
\r
169 > + * handler as quickly as possible, not retry it. */
\r
170 > + IGNORE_RESULT (write (2, msg, sizeof (msg) - 1));
\r
171 > + interrupted =3D 1;
\r
174 > +/* Like gethostname but guarantees that a null-terminated hostname is
\r
175 > + * returned, even if it has to make one up. Invalid characters are
\r
176 > + * substituted such that the hostname can be used within a filename.
\r
179 > +safe_gethostname (char *hostname, size_t len)
\r
183 > + if (gethostname (hostname, len) =3D=3D -1) {
\r
184 > + strncpy (hostname, "unknown", len);
\r
186 > + hostname[len - 1] =3D '\0';
\r
188 > + for (p =3D hostname; *p !=3D '\0'; p++) {
\r
189 > + if (*p =3D=3D '/' || *p =3D=3D ':')
\r
194 > +/* Call fsync() on a directory path. */
\r
195 > +static notmuch_bool_t
\r
196 > +sync_dir (const char *dir)
\r
198 > + notmuch_bool_t ret;
\r
201 > + fd =3D open (dir, O_RDONLY);
\r
202 > + if (fd =3D=3D -1) {
\r
203 > + fprintf (stderr, "Error: open() dir failed: %s\n", strerror (errno));
\r
206 > + ret =3D (fsync (fd) =3D=3D 0);
\r
208 > + fprintf (stderr, "Error: fsync() dir failed: %s\n", strerror (errno));
\r
214 > +/* Open a unique file in the 'tmp' sub-directory of dir.
\r
215 > + * Returns the file descriptor on success, or -1 on failure.
\r
216 > + * On success, file paths for the message in the 'tmp' and 'new'
\r
217 > + * directories are returned via tmppath and newpath,
\r
218 > + * and the path of the 'new' directory itself in newdir. */
\r
220 > +maildir_open_tmp_file (void *ctx, const char *dir,
\r
221 > + char **tmppath, char **newpath, char **newdir)
\r
224 > + char hostname[256];
\r
225 > + struct timeval tv;
\r
226 > + char *filename;
\r
229 > + /* We follow the Dovecot file name generation algorithm. */
\r
230 > + pid =3D getpid ();
\r
231 > + safe_gethostname (hostname, sizeof (hostname));
\r
233 > + gettimeofday (&tv, NULL);
\r
234 > + filename =3D talloc_asprintf (ctx, "%ld.M%ldP%d.%s",
\r
235 > + tv.tv_sec, tv.tv_usec, pid, hostname);
\r
236 > + if (! filename) {
\r
237 > + fprintf (stderr, "Out of memory\n");
\r
241 > + *tmppath =3D talloc_asprintf (ctx, "%s/tmp/%s", dir, filename);
\r
242 > + if (! *tmppath) {
\r
243 > + fprintf (stderr, "Out of memory\n");
\r
247 > + fd =3D open (*tmppath, O_WRONLY | O_CREAT | O_TRUNC | O_EXCL, 0600);
\r
248 > + } while (fd =3D=3D -1 && errno =3D=3D EEXIST);
\r
250 > + if (fd =3D=3D -1) {
\r
251 > + fprintf (stderr, "Error: opening %s: %s\n", *tmppath, strerror (errno));
\r
255 > + *newdir =3D talloc_asprintf (ctx, "%s/new", dir);
\r
256 > + *newpath =3D talloc_asprintf (ctx, "%s/new/%s", dir, filename);
\r
257 > + if (! *newdir || ! *newpath) {
\r
258 > + fprintf (stderr, "Out of memory\n");
\r
260 > + unlink (*tmppath);
\r
264 > + talloc_free (filename);
\r
269 > +/* Copy the contents of standard input (fdin) into fdout.
\r
270 > + * Returns TRUE if a non-empty file was written successfully.
\r
271 > + * Otherwise, return FALSE. */
\r
272 > +static notmuch_bool_t
\r
273 > +copy_stdin (int fdin, int fdout)
\r
275 > + notmuch_bool_t empty =3D TRUE;
\r
277 > + while (! interrupted) {
\r
278 > + ssize_t remain;
\r
279 > + char buf[4096];
\r
282 > + remain =3D read (fdin, buf, sizeof (buf));
\r
283 > + if (remain =3D=3D 0)
\r
285 > + if (remain < 0) {
\r
286 > + if (errno =3D=3D EINTR)
\r
288 > + fprintf (stderr, "Error: reading from standard input: %s\n",
\r
289 > + strerror (errno));
\r
295 > + ssize_t written =3D write (fdout, p, remain);
\r
296 > + if (written < 0 && errno =3D=3D EINTR)
\r
298 > + if (written <=3D 0) {
\r
299 > + fprintf (stderr, "Error: writing to temporary file: %s",
\r
300 > + strerror (errno));
\r
303 > + p +=3D written;
\r
304 > + remain -=3D written;
\r
305 > + empty =3D FALSE;
\r
306 > + } while (remain > 0);
\r
309 > + return (!interrupted && !empty);
\r
312 > +/* Add the specified message file to the notmuch database, applying tags.
\r
313 > + * The file is renamed to encode notmuch tags as maildir flags. */
\r
315 > +add_file_to_database (notmuch_database_t *notmuch, const char *path,
\r
316 > + tag_op_list_t *tag_ops)
\r
318 > + notmuch_message_t *message;
\r
319 > + notmuch_status_t status;
\r
321 > + status =3D notmuch_database_add_message (notmuch, path, &message);
\r
322 > + switch (status) {
\r
323 > + case NOTMUCH_STATUS_SUCCESS:
\r
324 > + case NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID:
\r
327 > + case NOTMUCH_STATUS_FILE_NOT_EMAIL:
\r
328 > + case NOTMUCH_STATUS_READ_ONLY_DATABASE:
\r
329 > + case NOTMUCH_STATUS_XAPIAN_EXCEPTION:
\r
330 > + case NOTMUCH_STATUS_OUT_OF_MEMORY:
\r
331 > + case NOTMUCH_STATUS_FILE_ERROR:
\r
332 > + case NOTMUCH_STATUS_NULL_POINTER:
\r
333 > + case NOTMUCH_STATUS_TAG_TOO_LONG:
\r
334 > + case NOTMUCH_STATUS_UNBALANCED_FREEZE_THAW:
\r
335 > + case NOTMUCH_STATUS_UNBALANCED_ATOMIC:
\r
336 > + case NOTMUCH_STATUS_LAST_STATUS:
\r
337 > + fprintf (stderr, "Error: failed to add `%s' to notmuch database: %s\n",
\r
338 > + path, notmuch_status_to_string (status));
\r
342 > + if (status =3D=3D NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID) {
\r
343 > + /* Don't change tags of an existing message. */
\r
344 > + status =3D notmuch_message_tags_to_maildir_flags (message);
\r
345 > + if (status !=3D NOTMUCH_STATUS_SUCCESS)
\r
346 > + fprintf (stderr, "Error: failed to sync tags to maildir flags\n");
\r
348 > + tag_op_list_apply (message, tag_ops, TAG_FLAG_MAILDIR_SYNC);
\r
351 > + notmuch_message_destroy (message);
\r
354 > +static notmuch_bool_t
\r
355 > +insert_message (void *ctx, notmuch_database_t *notmuch, int fdin,
\r
356 > + const char *dir, tag_op_list_t *tag_ops)
\r
362 > + char *cleanup_path;
\r
364 > + fdout =3D maildir_open_tmp_file (ctx, dir, &tmppath, &newpath, &newd=
\r
369 > + cleanup_path =3D tmppath;
\r
371 > + if (! copy_stdin (fdin, fdout))
\r
374 > + if (fsync (fdout) !=3D 0) {
\r
375 > + fprintf (stderr, "Error: fsync failed: %s\n", strerror (errno));
\r
382 > + /* Atomically move the new message file from the Maildir 'tmp' direc=
\r
384 > + * to the 'new' directory. We follow the Dovecot recommendation to
\r
385 > + * simply use rename() instead of link() and unlink().
\r
386 > + * See also: http://wiki.dovecot.org/MailboxFormat/Maildir#Mail_deli=
\r
389 > + if (rename (tmppath, newpath) !=3D 0) {
\r
390 > + fprintf (stderr, "Error: rename() failed: %s\n", strerror (errno));
\r
394 > + cleanup_path =3D newpath;
\r
396 > + if (! sync_dir (newdir))
\r
399 > + /* Even if adding the message to the notmuch database fails,
\r
400 > + * the message is on disk and we consider the delivery completed. */
\r
401 > + add_file_to_database (notmuch, newpath, tag_ops);
\r
406 > + if (fdout >=3D 0)
\r
408 > + unlink (cleanup_path);
\r
413 > +notmuch_insert_command (notmuch_config_t *config, int argc, char *argv[])
\r
415 > + notmuch_database_t *notmuch;
\r
416 > + struct sigaction action;
\r
417 > + const char *db_path;
\r
418 > + const char **new_tags;
\r
419 > + size_t new_tags_length;
\r
420 > + tag_op_list_t *tag_ops;
\r
421 > + char *query_string =3D NULL;
\r
422 > + const char *maildir;
\r
423 > + int opt_index =3D 1;
\r
424 > + unsigned int i;
\r
425 > + notmuch_bool_t ret;
\r
427 > + db_path =3D notmuch_config_get_database_path (config);
\r
428 > + new_tags =3D notmuch_config_get_new_tags (config, &new_tags_length);
\r
430 > + tag_ops =3D tag_op_list_create (config);
\r
431 > + if (tag_ops =3D=3D NULL) {
\r
432 > + fprintf (stderr, "Out of memory.\n");
\r
435 > + for (i =3D 0; i < new_tags_length; i++) {
\r
436 > + if (tag_op_list_append (tag_ops, new_tags[i], FALSE))
\r
440 > + if (parse_tag_command_line (config, argc - opt_index, argv + opt_ind=
\r
442 > + &query_string, tag_ops))
\r
445 > + if (*query_string !=3D '\0') {
\r
446 > + fprintf (stderr, "Error: unexpected query string: %s\n", query_string);
\r
450 > + maildir =3D db_path;
\r
452 > + /* Setup our handler for SIGINT. We do not set SA_RESTART so that co=
\r
454 > + * from standard input may be interrupted. */
\r
455 > + memset (&action, 0, sizeof (struct sigaction));
\r
456 > + action.sa_handler =3D handle_sigint;
\r
457 > + sigemptyset (&action.sa_mask);
\r
458 > + action.sa_flags =3D 0;
\r
459 > + sigaction (SIGINT, &action, NULL);
\r
461 > + if (notmuch_database_open (notmuch_config_get_database_path (config),
\r
462 > + NOTMUCH_DATABASE_MODE_READ_WRITE, ¬much))
\r
465 > + ret =3D insert_message (config, notmuch, STDIN_FILENO, maildir, tag_=
\r
468 > + notmuch_database_destroy (notmuch);
\r
470 > + return (ret) ? 0 : 1;
\r
472 > diff --git a/notmuch.c b/notmuch.c
\r
473 > index 45a73ce..d11f214 100644
\r
476 > @@ -44,6 +44,8 @@ static command_t commands[] =3D {
\r
477 > "Interactively setup notmuch for first use." },
\r
478 > { "new", notmuch_new_command, FALSE,
\r
479 > "Find and import new messages to the notmuch database." },
\r
480 > + { "insert", notmuch_insert_command, FALSE,
\r
481 > + "Add a new message into the maildir and notmuch database." },
\r
482 > { "search", notmuch_search_command, FALSE,
\r
483 > "Search for messages matching the given search terms." },
\r
484 > { "show", notmuch_show_command, FALSE,
\r
488 > _______________________________________________
\r
489 > notmuch mailing list
\r
490 > notmuch@notmuchmail.org
\r
491 > http://notmuchmail.org/mailman/listinfo/notmuch
\r