From 76c817476bda6426a45877ea6797d7d3554735a8 Mon Sep 17 00:00:00 2001 From: J Farkas Date: Sun, 3 Jan 2016 20:27:44 +0000 Subject: [PATCH] Re: [PATCH] cli/insert: do not lose the SMTP envelope --- 7c/1bc3ec7eaf2286d01482662bc7d0089017a8ab | 223 ++++++++++++++++++++++ 1 file changed, 223 insertions(+) create mode 100644 7c/1bc3ec7eaf2286d01482662bc7d0089017a8ab diff --git a/7c/1bc3ec7eaf2286d01482662bc7d0089017a8ab b/7c/1bc3ec7eaf2286d01482662bc7d0089017a8ab new file mode 100644 index 000000000..a2fe1ef50 --- /dev/null +++ b/7c/1bc3ec7eaf2286d01482662bc7d0089017a8ab @@ -0,0 +1,223 @@ +Return-Path: +X-Original-To: notmuch@notmuchmail.org +Delivered-To: notmuch@notmuchmail.org +Received: from localhost (localhost [127.0.0.1]) + by arlo.cworth.org (Postfix) with ESMTP id CCA1A6DE18FD + for ; Sun, 3 Jan 2016 12:27:52 -0800 (PST) +X-Virus-Scanned: Debian amavisd-new at cworth.org +X-Spam-Flag: NO +X-Spam-Score: 0.283 +X-Spam-Level: +X-Spam-Status: No, score=0.283 tagged_above=-999 required=5 tests=[AWL=0.086, + DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, + RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, + TVD_FROM_1=0.999] autolearn=disabled +Received: from arlo.cworth.org ([127.0.0.1]) + by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024) + with ESMTP id bEZjEEjPB331 for ; + Sun, 3 Jan 2016 12:27:49 -0800 (PST) +Received: from know-smtprelay-omc-1.server.virginmedia.net + (know-smtprelay-omc-1.server.virginmedia.net [80.0.253.65]) + by arlo.cworth.org (Postfix) with ESMTP id 178736DE18DD + for ; Sun, 3 Jan 2016 12:27:48 -0800 (PST) +Received: from dev.koan19.net ([82.1.197.255]) + by know-smtprelay-1-imp with bizsmtp + id 1YTl1s00E5X6CWA01YTlHh; Sun, 03 Jan 2016 20:27:45 +0000 +X-Originating-IP: [82.1.197.255] +X-Spam: 0 +X-Authority: v=2.1 cv=AJvf2gUA c=1 sm=1 tr=0 a=D+CNGfzuhY6ArhcYgadsyQ==:117 + a=D+CNGfzuhY6ArhcYgadsyQ==:17 a=jxr8AxaCAAAA:8 a=dmPqMsitAAAA:8 + a=hov-Noh0Y1sA:10 a=kj9zAlcOel0A:10 a=pGLkceISAAAA:8 a=uZvujYp8AAAA:8 + a=Mtf3CE5mHBFcJ1K1414A:9 a=CjuIK1q_8ugA:10 a=pFPAoXI41dMA:10 +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; + d=l2015aftruuq.dns007.net; i=@l2015aftruuq.dns007.net; q=dns/txt; s=l201512; + t=1451852864; h=To : Cc : Subject : References : MIME-Version : Content-Type + : In-Reply-To : From : Message-ID : Date : X-Originating-IP : Subject : From + : Date; bh=2TxZzTtcupPxkprsdWpezrUla+fEq+rzmfnIBTNbV3c=; + b=NN63EMpcsqL/y9w9Z8k01NlUqLdXxVFWzYq42xZPaIV4lqRxDWTugi49GkAg1bqbHtZe4L + RzTHWBJt+3D2dWC6Bvfk0zBngETg4pqTjzwQvJlgZIUVNGhGaCQCE0kquHxntFs24o7B9K3N + 93xWJnCi+Wnzj568Yb3dUYb7P6F3mlm+jeWEZMmux2z4k/x/s4EwJPq41vyD+sBruU+hcHmA == +To: Jani Nikula +Cc: notmuch@notmuchmail.org +Subject: Re: [PATCH] cli/insert: do not lose the SMTP envelope +References: <1451647279.42.86b0a8ab@201601.l2015aftruuq.dns007.net> + <87bn92wsx4.fsf@nikula.org> +MIME-Version: 1.0 +Content-Type: text/plain; charset=us-ascii +Content-Disposition: inline +In-Reply-To: <87bn92wsx4.fsf@nikula.org> +User-Agent: Mutt/1.5.23.1 (2014-03-12) +From: J Farkas +Message-ID: <1451852864.63.af86be8a@201601.l2015aftruuq.dns007.net> +Date: Sun, 03 Jan 2016 20:27:44 +0000 +X-BeenThere: notmuch@notmuchmail.org +X-Mailman-Version: 2.1.20 +Precedence: list +List-Id: "Use and development of the notmuch mail system." + +List-Unsubscribe: , + +List-Archive: +List-Post: +List-Help: +List-Subscribe: , + +X-List-Received-Date: Sun, 03 Jan 2016 20:27:52 -0000 + +On 2016-01-03 at 18:04:39, Jani Nikula wrote: +> On Fri, 01 Jan 2016, J Farkas wrote: +> > From: Janos Farkas +> > Subject: [PATCH] cli/insert: do not lose the SMTP envelope +> > +> > Make sure we store the envelope sender/recipient if provided by +> > qmail-command(8) in $RPLINE and $DTLINE. +> > --- +> > +> > I just realised that the messages delivered directly into maildir don't have +> > the usual envelope addresses that qmail provides. This is a piece of +> > information that's important to (at least my) troubleshooting, so I created a +> > patch that seems to work well, applies cleanly to master (and 0.21), and +> > provided a NEWS entry should it be necessary. +> +> I'd be more interested in seeing some tests for this... + +I was thinking of it, and it could be simply an assurance that the +functionality stays there after changes too. To be honest, the only +reason I didn't because the test suite is not passing in my environment, +either because of some gdb peculiarity, or some differences with emacs. + +Answering your comment about Mallory here -- the DTLINE and RPLINE are +practically qmail's way of splitting the first two headers that *should* +be written in the message, they can only be affected by someone who +is actually saying he wants to deliver a message, not for any other +deliveries. + +All the data that is supplied this way to the MDA by qmail (or the local +MTA), is still something that was accepted during the SMTP conversation +and passed basic checks, for a locally acceptable recipient, and after +any possible blocking on the sender. + +It's just done in this way because: + +- DTLINE is the Delivered-To line and qmail at this point is not sure + the file will be "delivered", or processed in some other way, only the + delivering program can actually tell what recipient will it be + delivered to. qmail uses this series of headers for loop avoidance, + so it's essential that all the checkpoints are present. + +- RPLINE is the Return-Path header that should be the *first* header in + the file; if it would become part of the stdio, now all delivering and + non-delivering programs would have to parse, detach it, and reattach + to the front after any headers added. + +It's basically a way to keep the SMTP conversation details alive along +the delivery pipeline. Throwing it away is incorrect. If this is not +ending up in the message, all I lose is the SMTP envelope, that can tell +me what entity was directly responsible to pass this message to my SMTP +server - was it sent by a mailing list, or if it's a direct message. +Or, in some cases, *which* mailing list. It's a much safer way than +parsing through the Received lines. + +Feel free to let me know if it needs further clarification why it is to +be done this way. + +> > NEWS | 9 +++++++++ +> > notmuch-insert.c | 28 ++++++++++++++++++++++++++++ +> > 2 files changed, 37 insertions(+) +> > +> > diff --git a/NEWS b/NEWS +> > index 6681699..13d45c8 100644 +> > --- a/NEWS +> > +++ b/NEWS +> > @@ -1,3 +1,12 @@ +> > + +> > + +> > +`notmuch insert` records the envelope addresses if available +> > + +> > + If the caller provides this information as qmail-command(8) does in +> > + the RPLINE and DTLINE environment variables, then notmuch insert will +> > + record it in the maildir file. +> +> We usually refer to message files. Perhaps you should also mention what +> the RPLINE and DTLINE variables should contain. + +I don't think it's worthy for a NEWS entry with an explanation for those +- perhaps you meant in the commit or comments? + +> > + +> > Notmuch 0.21 (2015-10-29) +> > ========================= +> > +> > diff --git a/notmuch-insert.c b/notmuch-insert.c +> > index 5205c17..ecc0fa0 100644 +> > --- a/notmuch-insert.c +> > +++ b/notmuch-insert.c +> > @@ -284,6 +284,26 @@ copy_fd (int fdout, int fdin) +> > } +> > +> > /* +> > + * Write zero (and LF) terminated string to the output fd. It's expected to +> > + * come from getenv(), so it's not checked for correctness. NULL or empty +> > + * string is ignored, successfully. +> > + * Return TRUE on success, FALSE on errors. +> > + */ +> > +static notmuch_bool_t +> > +write_header (int fdout, const char *hdr) +> > +{ +> > + ssize_t written,to_write; +> > + +> > + if (hdr && (to_write = strlen (hdr))) { +> > + written = write (fdout, hdr, to_write); +> > + if (written != to_write) +> > + return FALSE; +> > + } +> +> It's not an error for write() to return prematurely with written < +> to_write. Please see the write(2) man page and the copy_fd() +> implementation in this file. + +Yes, I considered it - I just couldn't see why any of the conditions +that can cause this, makes it worth to keep trying. + +My manuals, and even the POSIX write()* description is only mentioning +error conditions (end of medium, file size limits) and signals that can +cause a split write(). In case of a hard error (which can be resolved +some time later, for sure), the best choice is to abort it anyway. +There's no other signal that can divert the execution, just ctrl-c, in +which case the user is already expecting the program to quit. + +And - a "failure" in an MDA is not necessarily the worst way to handle +problems - the delivery is just deferred by the local queue, it will +only cause an error if it was persistently failing for a long time. + +But if you think it should be more robust at this point, I'll be happy +to redo the error handling as expected. + +* http://pubs.opengroup.org/onlinepubs/9699919799/functions/write.html + + +> Jani. +> +> > + +> > + return TRUE; +> > +} +> > + +> > +/* +> > * Write fdin to a new temp file in maildir/tmp, return full path to +> > * the file, or NULL on errors. +> > */ +> > @@ -297,6 +317,14 @@ maildir_write_tmp (const void *ctx, int fdin, const char *maildir) +> > if (fdout < 0) +> > return NULL; +> > +> > + /* maildir(5) suggests the message should start with a Return-Path +> > + * and Delivered-To lines. qmail-local(8) supplies these. +> > + */ +> > + if (! write_header(fdout, getenv("RPLINE"))) +> > + goto FAIL; +> > + if (! write_header(fdout, getenv("DTLINE"))) +> > + goto FAIL; +> > + +> > if (! copy_fd (fdout, fdin)) +> > goto FAIL; + -- 2.26.2