--- /dev/null
+Return-Path: <teythoon@jade-hamburg.de>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+ by olra.theworths.org (Postfix) with ESMTP id 16C6C431FBD\r
+ for <notmuch@notmuchmail.org>; Mon, 24 Jun 2013 01:57:44 -0700 (PDT)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: 0\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=0 tagged_above=-999 required=5 tests=[none]\r
+ autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+ by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+ with ESMTP id d+Fd0wHck8ug for <notmuch@notmuchmail.org>;\r
+ Mon, 24 Jun 2013 01:57:34 -0700 (PDT)\r
+Received: from mail.cryptobitch.de (cryptobitch.de [88.198.7.68])\r
+ (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))\r
+ (No client certificate requested)\r
+ by olra.theworths.org (Postfix) with ESMTPS id 8B5F8431FAF\r
+ for <notmuch@notmuchmail.org>; Mon, 24 Jun 2013 01:57:34 -0700 (PDT)\r
+Received: from mail.jade-hamburg.de (mail.jade-hamburg.de [85.183.11.228])\r
+ (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))\r
+ (No client certificate requested)\r
+ by mail.cryptobitch.de (Postfix) with ESMTPSA id 0509662C358\r
+ for <notmuch@notmuchmail.org>; Mon, 24 Jun 2013 10:57:17 +0200 (CEST)\r
+Received: by mail.jade-hamburg.de (Postfix, from userid 401)\r
+ id 812A4DF2A5; Mon, 24 Jun 2013 10:57:15 +0200 (CEST)\r
+Received: from thinkbox.jade-hamburg.de (unknown\r
+ [IPv6:2002:55b7:be4:1:216:d3ff:fe3e:5058])\r
+ (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits))\r
+ (No client certificate requested) (Authenticated sender: teythoon)\r
+ by mail.jade-hamburg.de (Postfix) with ESMTPSA id 99CADDF29F;\r
+ Mon, 24 Jun 2013 10:57:11 +0200 (CEST)\r
+Received: from teythoon by thinkbox.jade-hamburg.de with local (Exim 4.80)\r
+ (envelope-from <teythoon@thinkbox.jade-hamburg.de>)\r
+ id 1Ur2aE-0008Nc-K6; Mon, 24 Jun 2013 10:57:10 +0200\r
+Content-Type: text/plain; charset="utf-8"\r
+MIME-Version: 1.0\r
+Content-Transfer-Encoding: quoted-printable\r
+To: Austin Clements <amdragon@MIT.EDU>,\r
+ thomas schwinge <thomas@codesourcery.com>, \r
+From: Justus Winter <4winter@informatik.uni-hamburg.de>\r
+In-Reply-To: <20130623165938.GA2214@mit.edu>\r
+References: <20130623131145.2526.439@thinkbox.jade-hamburg.de>\r
+ <20130623165938.GA2214@mit.edu>\r
+Message-ID: <20130624085710.31827.41792@thinkbox.jade-hamburg.de>\r
+User-Agent: alot/0.3.4\r
+Subject: Re: header continuation issue in notmuch frontend/alot/pythons email\r
+ module\r
+Date: Mon, 24 Jun 2013 10:57:10 +0200\r
+Cc: notmuch mailing list <notmuch@notmuchmail.org>\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+ <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Mon, 24 Jun 2013 08:57:44 -0000\r
+\r
+Quoting Austin Clements (2013-06-23 18:59:39)\r
+> Quoth Justus Winter on Jun 23 at 3:11 pm:\r
+> > Hi,\r
+> > =\r
+\r
+> > I recently had a problem replying to a mail written by Thomas Schwinge\r
+> > using an oldish notmuch. Not sure if it has been fixed in more recent\r
+> > versions, but I think notmuch could improve uppon its header\r
+> > generation (see below). Problematic part of the mail:\r
+> > =\r
+\r
+> > ~~~ snip ~~~\r
+> > [...]\r
+> > To: someone@example.org, "line\r
+> > break" <linebreak@example.org>, someoneelse@example.org\r
+> > User-Agent: Notmuch/0.9-101-g81dad07 (http://notmuchmail.org) Emacs/23.=\r
+4.1 (i486-pc-linux-gnu)\r
+> > [...]\r
+> > ~~~ snap ~~~\r
+> > =\r
+\r
+> > http://tools.ietf.org/html/rfc2822#section-2.2.3 says:\r
+> > =\r
+\r
+> > Note: Though structured field bodies are defined in such a way that\r
+> > folding can take place between many of the lexical tokens (and even\r
+> > within some of the lexical tokens), folding SHOULD be limited to\r
+> > placing the CRLF at higher-level syntactic breaks. For instance, if\r
+> > a field body is defined as comma-separated values, it is recommended\r
+> > that folding occur after the comma separating the structured items in\r
+> > preference to other places where the field could be folded, even if\r
+> > it is allowed elsewhere.\r
+> > =\r
+\r
+> > So notmuch "rfc-SHOULD" place the newlines after the comma.\r
+> > =\r
+\r
+> > The rfc goes on:\r
+> > =\r
+\r
+> > The process of moving from this folded multiple-line representation\r
+> > of a header field to its single line representation is called\r
+> > "unfolding". Unfolding is accomplished by simply removing any CRLF\r
+> > that is immediately followed by WSP. Each header field should be\r
+> > treated in its unfolded form for further syntactic and semantic\r
+> > evaluation.\r
+> > =\r
+\r
+> > My interpretation is that unfolding simply removes any linebreaks\r
+> > first, so the value does not contain any newlines. But pythons email\r
+> > module discriminates quoted and unquoted parts of the value:\r
+> > =\r
+\r
+> > ~~~ snip ~~~\r
+> > from __future__ import print_function\r
+> > import email\r
+> > from email.utils import getaddresses\r
+> > =\r
+\r
+> > m =3D email.message_from_string('''To: "line\r
+> > break" <linebreak@example.org>, line\r
+> > break <linebreak@example.org>''')\r
+> > print("m['To'] =3D ", m['To'])\r
+> > print("getaddresses(m.get_all('To')) =3D ", getaddresses(m.get_all('To'=\r
+)))\r
+> > ~~~ snap ~~~\r
+> > =\r
+\r
+> > % python3 test.py\r
+> > m['To'] =3D "line\r
+> > break" <linebreak@example.org>, line\r
+> > break <linebreak@example.org>\r
+> > getaddresses(m.get_all('To')) =3D [('line\n break', 'linebreak@example=\r
+.org'), ('line break', 'linebreak@example.org')]\r
+> > =\r
+\r
+> > I believe that is what's preventing me from replying to the message\r
+> > using alot without sanitizing the To header first. Not really sure who\r
+> > is wrong or right here... any thoughts?\r
+> =\r
+\r
+> There are at least two bugs here. Regardless of what we RFC-should\r
+> do, that folding *is* permitted by RFC2822, since quoted\r
+> strings can contain folding whitespace:\r
+> =\r
+\r
+> http://tools.ietf.org/html/rfc2822#section-3.2.5\r
+> =\r
+\r
+> For completeness, the full derivation for this "To" header is:\r
+> =\r
+\r
+> to =3D "To:" address-list CRLF\r
+> address-list =3D (address *("," address)) / obs-addr-list\r
+> address =3D mailbox / group\r
+> mailbox =3D name-addr / addr-spec\r
+> name-addr =3D [display-name] angle-addr\r
+> display-name =3D phrase\r
+> phrase =3D 1*word / obs-phrase\r
+> word =3D atom / quoted-string\r
+> quoted-string =3D [CFWS]\r
+> DQUOTE *([FWS] qcontent) [FWS] DQUOTE\r
+> [CFWS]\r
+> =\r
+\r
+> Do you happen to know how the strangely folded "to" header was\r
+> produced for this message?\r
+\r
+No, but Thomas might. Thomas, the problematic message is\r
+id:877ghpqckb.fsf@kepler.schwinge.homeip.net\r
+\r
+> In notmuch-emacs, a user can put whatever\r
+> they want in a message-mode buffer's headers and mm will dutifully\r
+> pass it on to their MTA. We could validate it, but that's a slippery\r
+> slope and I would hope that the MTA itself is validating it (and\r
+> probably more thoroughly than we could).\r
+> =\r
+\r
+> That said, the first bug here is in Python. As I mentioned above,\r
+> foldable whitespace is allowed in quoted strings. In fact, though the\r
+> standard is rather long-winded about whitespace, if you dig into the\r
+> grammar, you'll find that *all whitespace can be folded* (except in\r
+> the obsolete grammar, which allowed whitespace between the header name\r
+> and the colon, which obviously can't be folded). I'm not sure what\r
+> Python is doing, but I bet it's going to a lot of effort to\r
+> mis-implement something very simple.\r
+\r
+Yes, I'm glad you came to the same conclusion.\r
+\r
+> There also appears to be a bug in the notmuch CLI's reply command\r
+> where it omits addresses that were folded in the original message. I\r
+> don't know if alot uses the CLI's reply command, so this may or may\r
+> not be related to your specific issue. I haven't dug into this yet,\r
+> other than to confirm that it's the CLI's fault and not\r
+> notmuch-emacs's.\r
+\r
+No, alot does not use notmuchs reply command.\r
+\r
+Thanks,\r
+Justus\r
projects / notmuch-archives.git / commitdiff