Re: emacs complains about encoding?

diff --git a/c7/cf1430b3ddde896a857ce1a6e071947314c57f b/c7/cf1430b3ddde896a857ce1a6e071947314c57f
new file mode 100644 (file)
index 0000000..d247ada
--- /dev/null
+++ b/c7/cf1430b3ddde896a857ce1a6e071947314c57f
@@ -0,0 +1,142 @@
+Return-Path: <awg@xvx.ca>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+       by olra.theworths.org (Postfix) with ESMTP id 7F509431FB6\r
+       for <notmuch@notmuchmail.org>; Sun, 20 May 2012 08:34:56 -0700 (PDT)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -0.7\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-0.7 tagged_above=-999 required=5\r
+       tests=[RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+       by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+       with ESMTP id qYJUBtrE5ElG for <notmuch@notmuchmail.org>;\r
+       Sun, 20 May 2012 08:34:54 -0700 (PDT)\r
+Received: from mail-lpp01m010-f53.google.com (mail-lpp01m010-f53.google.com\r
+       [209.85.215.53]) (using TLSv1 with cipher RC4-SHA (128/128 bits))\r
+       (No client certificate requested)\r
+       by olra.theworths.org (Postfix) with ESMTPS id 978E8431FAE\r
+       for <notmuch@notmuchmail.org>; Sun, 20 May 2012 08:34:54 -0700 (PDT)\r
+Received: by lagu2 with SMTP id u2so3478750lag.26\r
+       for <notmuch@notmuchmail.org>; Sun, 20 May 2012 08:34:51 -0700 (PDT)\r
+X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;\r
+       d=google.com; s=20120113;\r
+       h=mime-version:sender:x-originating-ip:in-reply-to:references:date\r
+       :x-google-sender-auth:message-id:subject:from:to:cc:content-type\r
+       :content-transfer-encoding:x-gm-message-state;\r
+       bh=tEWgpW06KE7GKjpnHkKqwR6tdvV5SURNuhRR+Ip755g=;\r
+       b=mWdfREXlujHWx4OqhubSvZ9qyPDwn8N3AFiGx0otsmKvO/gnTqNK53tRppjyf+QcbJ\r
+       2UvciQtw15+LKACSrk8PA55RcC29LStdySLI5VCDu46jBb7ZwaFiZ3wYJmKP6aB2KDES\r
+       ZANG+nqZvkvqSVMmOpa/5O2qMxIhgeW2hQipfIYiSXWu/qVZLBAnn13ax2+hVL7/Oj3U\r
+       WCiIjAj2DySaird7ecNywVZpycOtylGFjBCuv39Ei/n9NHA8pne6enSyxh93CwUoxvTM\r
+       Iazjcxe+oO0Pdskz+DnjJUHNY1TxldTvmhgA28NJIsfGYWlBXNnkP8MgmolpeAt/1egS\r
+       G2dQ==\r
+MIME-Version: 1.0\r
+Received: by 10.152.145.41 with SMTP id sr9mr10681666lab.25.1337528091518;\r
+       Sun, 20 May 2012 08:34:51 -0700 (PDT)\r
+Sender: awg@xvx.ca\r
+Received: by 10.112.82.163 with HTTP; Sun, 20 May 2012 08:34:51 -0700 (PDT)\r
+X-Originating-IP: [96.52.216.56]\r
+In-Reply-To: <m23970bhre.fsf@guru.guru-group.fi>\r
+References: <20120515194455.B7AD5100646@guru.guru-group.fi>\r
+       <878vgsbprq.fsf@nikula.org> <m23970bhre.fsf@guru.guru-group.fi>\r
+Date: Sun, 20 May 2012 09:34:51 -0600\r
+X-Google-Sender-Auth: 92fN3P7EpdBUqCWr4e1SsCOyQ-M\r
+Message-ID:\r
+ <CAMoJFUungAFPWy0d1Lh+rqmpK--P7MMEwNaewWHR=rbYo+BKsA@mail.gmail.com>\r
+Subject: Re: emacs complains about encoding?\r
+From: Adam Wolfe Gordon <awg+notmuch@xvx.ca>\r
+To: Tomi Ollila <tomi.ollila@iki.fi>\r
+Content-Type: text/plain; charset=ISO-8859-1\r
+Content-Transfer-Encoding: quoted-printable\r
+X-Gm-Message-State:\r
+ ALoCoQnZSxpxAMqJqUbF4AsRCO4a+i0bJBP47PyxcuUvQUIOciwUWyMlj/Lv7JR4HFwQkzhoFtyv\r
+Cc: notmuch@notmuchmail.org\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+       <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Sun, 20 May 2012 15:34:56 -0000\r
+\r
+On Wed, May 16, 2012 at 3:24 AM, Tomi Ollila <tomi.ollila@iki.fi> wrote:\r
+> Haa, It doesn't matter which is the original encoding of the message;\r
+>\r
+> notmuch reply id:20120515194455.B7AD5100646@guru.guru-group.fi\r
+>\r
+> where =A0notmuch show --format=3Draw ^^^ =A0outputs (among other lines):\r
+>\r
+> =A0Content-Type: text/plain; charset=3D"iso-8859-1"\r
+> =A0Content-Transfer-Encoding: quoted-printable\r
+>\r
+> and\r
+>\r
+> notmuch reply id:"878vgsbprq.fsf@nikula.org"\r
+>\r
+> where =A0notmuch show --format=3Draw ^^^ =A0outputs (among other lines):\r
+>\r
+> =A0Content-Type: text/plain; charset=3D"utf-8"\r
+> =A0Content-Transfer-Encoding: base64\r
+>\r
+> produce correct reply content, both in utf-8.\r
+>\r
+> So it is the emacs side which breaks replies.\r
+\r
+It turns out it's actually not the emacs side, but an interaction\r
+between our JSON reply format and emacs.\r
+\r
+The JSON reply (and show) code includes part content for all text/*\r
+parts except text/html. Because all JSON is required to be UTF-8, it\r
+handles the encoding itself, puts UTF-8 text in, and omits a\r
+content-charset field from the output. Emacs passes on the\r
+content-charset field to mm-display-part-inline if it's available, but\r
+for text/plain parts it's not, leaving mm-display-part-inline to its\r
+own devices for figuring out what the charset is. It seems\r
+mm-display-part-inline correctly figures out that it's UTF-8, and puts\r
+in the series of ugly \nnn characters because that's what emacs does\r
+with UTF-8 sometimes.\r
+\r
+In the original reply stuff (pre-JSON reply format) emacs used the\r
+output of notmuch reply verbatim, so all the charset stuff was handled\r
+in notmuch. Before f6c170fabca8f39e74705e3813504137811bf162, emacs was\r
+using the JSON reply format, but was inserting the text itself instead\r
+of using mm-display-part-inline, so emacs still wasn't trying to do\r
+any charset manipulation. Using mm-display-part-inline is desirable\r
+because it lets us handle non-text/plain (e.g. text/html) parts\r
+correctly in reply, and makes the display more consistent (since we\r
+use it for show). But, it leads to this problem.\r
+\r
+So, there are a couple of solutions I can see:\r
+\r
+1) Have the JSON formats include the original content-charset even\r
+though they're actually outputting UTF-8. Of the solutions I tried,\r
+this is the best, even though it doesn't sound like a good thing to\r
+do.\r
+\r
+2) Have the JSON formats include content only if it's actually UTF-8.\r
+This means that for non-UTF-8 parts (including ASCII parts), the emacs\r
+interface has to do more work to display the part content, since it\r
+must fetch it from outside first. When I tried this, it worked but\r
+caused the \nnn to show up when viewing messages in emacs. I suspect\r
+this is because it sets a charset for the whole buffer, and can't\r
+accommodate messages with different charsets in the same buffer\r
+properly. Reply works correctly, though.\r
+\r
+3) Have the JSON formats include the charset for all parts, but make\r
+it UTF-8 for all parts they include content for (since we're actually\r
+outputting UTF-8). This doesn't seem to fix the problem, even though\r
+it seems like it should.\r
+\r
+If no one has a better idea or a strong reason not to, I'll send a\r
+patch for solution (1).\r
+\r
+-- Adam\r