From: Peter Wang Date: Tue, 28 May 2013 13:58:55 +0000 (+1000) Subject: Re: [PATCH v5 03/12] cli: add insert command X-Git-Url: http://git.tremily.us/?a=commitdiff_plain;h=ea520094eb26efa463da54f15cf653b0f73609a1;p=notmuch-archives.git Re: [PATCH v5 03/12] cli: add insert command --- diff --git a/c2/163bd379a00b25d938d456cc10ad380a7c57ab b/c2/163bd379a00b25d938d456cc10ad380a7c57ab new file mode 100644 index 000000000..2621ba5da --- /dev/null +++ b/c2/163bd379a00b25d938d456cc10ad380a7c57ab @@ -0,0 +1,208 @@ +Return-Path: +X-Original-To: notmuch@notmuchmail.org +Delivered-To: notmuch@notmuchmail.org +Received: from localhost (localhost [127.0.0.1]) + by olra.theworths.org (Postfix) with ESMTP id 76529431FAF + for ; Tue, 28 May 2013 06:59:14 -0700 (PDT) +X-Virus-Scanned: Debian amavisd-new at olra.theworths.org +X-Spam-Flag: NO +X-Spam-Score: -0.099 +X-Spam-Level: +X-Spam-Status: No, score=-0.099 tagged_above=-999 required=5 + tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, + FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001] autolearn=disabled +Received: from olra.theworths.org ([127.0.0.1]) + by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) + with ESMTP id oJFqyr+g1NNd for ; + Tue, 28 May 2013 06:59:04 -0700 (PDT) +Received: from mail-pd0-f176.google.com (mail-pd0-f176.google.com + [209.85.192.176]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) + (No client certificate requested) + by olra.theworths.org (Postfix) with ESMTPS id 87309431FAE + for ; Tue, 28 May 2013 06:59:04 -0700 (PDT) +Received: by mail-pd0-f176.google.com with SMTP id r11so7618348pdi.21 + for ; Tue, 28 May 2013 06:59:02 -0700 (PDT) +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; + h=date:message-id:from:to:cc:subject:in-reply-to:references + :mime-version:content-type:content-disposition + :content-transfer-encoding; + bh=8lUzYxPqsl9CVde2gMByvB4r4rbOjTr/fHQOV7ys2fc=; + b=0Czw2bvxjImWP4KjtxwoJhdMH8xp9ZEXUwQBukr+3VnIq13FxnK3JdWmaHCHiOTHDS + /MVZYccjaEzw+CkIOzo+HTxqbD/MPZzEin0GlgTfcv5A2Tvoyxs1/1B3J7R86PbLdZjB + xqFzBsl9ZYxu6nWh4w7UBuegGCL/E2S78c/eFdhbywkhUwSpMvR3dzpn/YTSk4VCq2z4 + IVaNMj6zXnWZyEHZaYrb2aEnSQZfIE8YiAtNK4MS8cYB3q5+W9cEXCKdHeXwDApo6gt7 + Ltn0g0p+Tdb75tqmc37/GzDvEqmuoBul7yx8J4gsvBbcd7Ld8ogDmA4kbBvUbbeq8Svm + gHuw== +X-Received: by 10.68.222.74 with SMTP id qk10mr34022999pbc.58.1369749542622; + Tue, 28 May 2013 06:59:02 -0700 (PDT) +Received: from localhost (215.42.233.220.static.exetel.com.au. + [220.233.42.215]) by mx.google.com with ESMTPSA id + cp1sm32982475pbc.42.2013.05.28.06.58.58 for + (version=TLSv1.2 cipher=RC4-SHA bits=128/128); + Tue, 28 May 2013 06:59:01 -0700 (PDT) +Date: Tue, 28 May 2013 23:58:55 +1000 +Message-ID: <20130528235855.GB3209@hili.localdomain> +From: Peter Wang +To: Jani Nikula +Subject: Re: [PATCH v5 03/12] cli: add insert command +In-Reply-To: <8761z7r7jn.fsf@nikula.org> +References: <1364942517-6982-1-git-send-email-novalazy@gmail.com> + <1364942517-6982-4-git-send-email-novalazy@gmail.com> + <8761z7r7jn.fsf@nikula.org> +MIME-Version: 1.0 +Content-Type: text/plain; charset=utf-8 +Content-Disposition: inline +Content-Transfer-Encoding: 8bit +Cc: notmuch@notmuchmail.org +X-BeenThere: notmuch@notmuchmail.org +X-Mailman-Version: 2.1.13 +Precedence: list +List-Id: "Use and development of the notmuch mail system." + +List-Unsubscribe: , + +List-Archive: +List-Post: +List-Help: +List-Subscribe: , + +X-List-Received-Date: Tue, 28 May 2013 13:59:14 -0000 + +On Sun, 28 Apr 2013 00:24:28 +0300, Jani Nikula wrote: +> On Wed, 03 Apr 2013, Peter Wang wrote: +> > The notmuch insert command reads a message from standard input, +> > writes it to a Maildir folder, and then incorporates the message into +> > the notmuch database. Essentially it moves the functionality of +> > notmuch-deliver into notmuch. +> > +> > Though it could be used as an alternative to notmuch new, the reason +> > I want this is to allow my notmuch frontend to add postponed or sent +> > messages to the mail store and notmuch database, without resorting to +> > another tool (e.g. notmuch-deliver) nor directly modifying the maildir. +> > --- +> > Makefile.local | 1 + +> > notmuch-client.h | 3 + +> > notmuch-insert.c | 336 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ +> > notmuch.c | 3 + +> > 4 files changed, 343 insertions(+) +> > create mode 100644 notmuch-insert.c +> > +... +> > +/* Add the specified message file to the notmuch database, applying tags. +> > + * The file is renamed to encode notmuch tags as maildir flags. */ +> > +static notmuch_bool_t +> > +add_file_to_database (notmuch_database_t *notmuch, const char *path, +> > + tag_op_list_t *tag_ops) +> > +{ +> > + notmuch_message_t *message; +> > + notmuch_status_t status; +> > + +> > + status = notmuch_database_add_message (notmuch, path, &message); +> > + switch (status) { +> > + case NOTMUCH_STATUS_SUCCESS: +> > + case NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID: +> > + break; +> > + default: +> > + case NOTMUCH_STATUS_FILE_NOT_EMAIL: +> +> If such a message really arrives, the mail system will keep trying if +> failure is returned. Maybe deliver the file without indexing, and return +> success? +> + +Rethinking it, if notmuch insert is going to used as a general mail +delivery tool (not my own use case) then its primary job should be to +get the file to disk. As long as that is done, we should return success. + +Indexing the message would be considered a bonus, and failure there +or in syncing tags to flags should not cause the file to be deleted and +an error code returned. (A warning can be written to standard error.) + +> > + case NOTMUCH_STATUS_READ_ONLY_DATABASE: +> > + case NOTMUCH_STATUS_XAPIAN_EXCEPTION: +> > + case NOTMUCH_STATUS_OUT_OF_MEMORY: +> > + case NOTMUCH_STATUS_FILE_ERROR: +> > + case NOTMUCH_STATUS_NULL_POINTER: +> > + case NOTMUCH_STATUS_TAG_TOO_LONG: +> > + case NOTMUCH_STATUS_UNBALANCED_FREEZE_THAW: +> > + case NOTMUCH_STATUS_UNBALANCED_ATOMIC: +> > + case NOTMUCH_STATUS_LAST_STATUS: +> > + fprintf (stderr, "Error: failed to add `%s' to notmuch database: %s\n", +> > + path, notmuch_status_to_string (status)); +> > + return FALSE; +> > + } +> > + +> > + if (status == NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID) { +> > + /* Don't change tags of an existing message. */ +> > + status = notmuch_message_tags_to_maildir_flags (message); +> > + if (status != NOTMUCH_STATUS_SUCCESS) +> > + fprintf (stderr, "Error: failed to sync tags to maildir flags\n"); +> > + } else { +> > + status = tag_op_list_apply (message, tag_ops, TAG_FLAG_MAILDIR_SYNC); +> +> Syncing tags to maildir flags is more interesting here than above. And +> it should be done because notmuch insert allows arbitrary tags on the +> command line. Having, for example, -unread or +flagged on the command +> line makes the flags go out of sync. (notmuch new should do the syncing +> too, but it's less important because it only adds new.tags.) +> +> However, calling notmuch_message_tags_to_maildir_flags() may rename the +> file from new to cur, which blows up the directory syncing and file +> unlinking on the error path in insert_message() below. + +We would sidestep these problems. + +> > +static notmuch_bool_t +> > +insert_message (void *ctx, notmuch_database_t *notmuch, int fdin, +> > + const char *dir, tag_op_list_t *tag_ops) +> > +{ +> > + char *tmppath; +> > + char *newpath; +> > + char *newdir; +> > + int fdout; +> > + char *cleanup_path; +> > + +> > + fdout = maildir_open_tmp_file (ctx, dir, &tmppath, &newpath, &newdir); +> > + if (fdout < 0) +> > + return FALSE; +> > + +> > + cleanup_path = tmppath; +> > + +> > + if (! copy_stdin (fdin, fdout)) +> > + goto FAIL; +> > + +> > + if (fsync (fdout) != 0) { +> > + fprintf (stderr, "Error: fsync failed: %s\n", strerror (errno)); +> > + goto FAIL; +> > + } +> > + +> > + close (fdout); +> > + fdout = -1; +> > + +> > + /* Atomically move the new message file from the Maildir 'tmp' directory +> > + * to the 'new' directory. We follow the Dovecot recommendation to +> > + * simply use rename() instead of link() and unlink(). +> > + * See also: http://wiki.dovecot.org/MailboxFormat/Maildir#Mail_delivery +> > + */ +> > + if (rename (tmppath, newpath) != 0) { +> > + fprintf (stderr, "Error: rename() failed: %s\n", strerror (errno)); +> > + goto FAIL; +> > + } +> > + +> > + cleanup_path = newpath; +> > + +> > + if (! add_file_to_database (notmuch, newpath, tag_ops)) { +> > + /* XXX add an option to keep the file in maildir? */ +> +> Possibly a good idea to let the user decide. This is the part that I +> worry most about in the series. It's not unusual to hit xapian +> exceptions when the database is already locked, which results +> (hopefully!) in another attempt at delivery. If not, mail is lost. +> +> However, keeping the file in maildir on indexing errors ignores the tags +> specified on the notmuch insert command line, and the message will get +> just new.tags next time notmuch new is run. This is not nice either. + +Not sure what else could be done. + +Peter