1 Return-Path: <sojka@os.inf.tu-dresden.de>
\r
2 X-Original-To: notmuch@notmuchmail.org
\r
3 Delivered-To: notmuch@notmuchmail.org
\r
4 Received: from localhost (localhost [127.0.0.1])
\r
5 by olra.theworths.org (Postfix) with ESMTP id E6C48431FB6
\r
6 for <notmuch@notmuchmail.org>; Tue, 22 May 2012 05:53:35 -0700 (PDT)
\r
7 X-Virus-Scanned: Debian amavisd-new at olra.theworths.org
\r
11 X-Spam-Status: No, score=-2.3 tagged_above=-999 required=5
\r
12 tests=[RCVD_IN_DNSWL_MED=-2.3] autolearn=disabled
\r
13 Received: from olra.theworths.org ([127.0.0.1])
\r
14 by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)
\r
15 with ESMTP id 6EZeAtj3uhqd for <notmuch@notmuchmail.org>;
\r
16 Tue, 22 May 2012 05:53:34 -0700 (PDT)
\r
17 Received: from max.feld.cvut.cz (max.feld.cvut.cz [147.32.192.36])
\r
18 by olra.theworths.org (Postfix) with ESMTP id 9C76B431FAE
\r
19 for <notmuch@notmuchmail.org>; Tue, 22 May 2012 05:53:34 -0700 (PDT)
\r
20 Received: from localhost (unknown [192.168.200.4])
\r
21 by max.feld.cvut.cz (Postfix) with ESMTP id 2D88919F3345;
\r
22 Tue, 22 May 2012 14:53:33 +0200 (CEST)
\r
23 X-Virus-Scanned: IMAP AMAVIS
\r
24 Received: from max.feld.cvut.cz ([192.168.200.1])
\r
25 by localhost (styx.feld.cvut.cz [192.168.200.4]) (amavisd-new,
\r
27 with ESMTP id AYx5Np-t01_W; Tue, 22 May 2012 14:53:28 +0200 (CEST)
\r
28 Received: from imap.feld.cvut.cz (imap.feld.cvut.cz [147.32.192.34])
\r
29 by max.feld.cvut.cz (Postfix) with ESMTP id 4839919F331A;
\r
30 Tue, 22 May 2012 14:53:28 +0200 (CEST)
\r
31 Received: from steelpick.2x.cz (note-sojka.felk.cvut.cz [147.32.86.30])
\r
32 (Authenticated sender: sojkam1)
\r
33 by imap.feld.cvut.cz (Postfix) with ESMTPSA id 2164E660968;
\r
34 Tue, 22 May 2012 14:53:27 +0200 (CEST)
\r
35 Received: from wsh by steelpick.2x.cz with local (Exim 4.77)
\r
36 (envelope-from <sojka@os.inf.tu-dresden.de>)
\r
37 id 1SWoac-0000JS-V4; Tue, 22 May 2012 14:53:26 +0200
\r
38 From: Michal Sojka <sojkam1@fel.cvut.cz>
\r
39 To: Adam Wolfe Gordon <awg+notmuch@xvx.ca>, Tomi Ollila <tomi.ollila@iki.fi>
\r
40 Subject: Re: emacs complains about encoding?
\r
42 <CAMoJFUungAFPWy0d1Lh+rqmpK--P7MMEwNaewWHR=rbYo+BKsA@mail.gmail.com>
\r
43 References: <20120515194455.B7AD5100646@guru.guru-group.fi>
\r
44 <878vgsbprq.fsf@nikula.org> <m23970bhre.fsf@guru.guru-group.fi>
\r
45 <CAMoJFUungAFPWy0d1Lh+rqmpK--P7MMEwNaewWHR=rbYo+BKsA@mail.gmail.com>
\r
46 User-Agent: Notmuch/0.12+185~g9826d2c (http://notmuchmail.org) Emacs/23.4.1
\r
47 (x86_64-pc-linux-gnu)
\r
48 Date: Tue, 22 May 2012 14:53:26 +0200
\r
49 Message-ID: <871umc1int.fsf@steelpick.2x.cz>
\r
51 Content-Type: text/plain; charset=us-ascii
\r
52 Cc: notmuch@notmuchmail.org
\r
53 X-BeenThere: notmuch@notmuchmail.org
\r
54 X-Mailman-Version: 2.1.13
\r
56 List-Id: "Use and development of the notmuch mail system."
\r
57 <notmuch.notmuchmail.org>
\r
58 List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,
\r
59 <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>
\r
60 List-Archive: <http://notmuchmail.org/pipermail/notmuch>
\r
61 List-Post: <mailto:notmuch@notmuchmail.org>
\r
62 List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>
\r
63 List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,
\r
64 <mailto:notmuch-request@notmuchmail.org?subject=subscribe>
\r
65 X-List-Received-Date: Tue, 22 May 2012 12:53:36 -0000
\r
69 Adam Wolfe Gordon <awg+notmuch@xvx.ca> writes:
\r
70 > It turns out it's actually not the emacs side, but an interaction
\r
71 > between our JSON reply format and emacs.
\r
73 > The JSON reply (and show) code includes part content for all text/*
\r
74 > parts except text/html. Because all JSON is required to be UTF-8, it
\r
75 > handles the encoding itself, puts UTF-8 text in, and omits a
\r
76 > content-charset field from the output. Emacs passes on the
\r
77 > content-charset field to mm-display-part-inline if it's available, but
\r
78 > for text/plain parts it's not, leaving mm-display-part-inline to its
\r
79 > own devices for figuring out what the charset is. It seems
\r
80 > mm-display-part-inline correctly figures out that it's UTF-8, and puts
\r
81 > in the series of ugly \nnn characters because that's what emacs does
\r
82 > with UTF-8 sometimes.
\r
84 > In the original reply stuff (pre-JSON reply format) emacs used the
\r
85 > output of notmuch reply verbatim, so all the charset stuff was handled
\r
86 > in notmuch. Before f6c170fabca8f39e74705e3813504137811bf162, emacs was
\r
87 > using the JSON reply format, but was inserting the text itself instead
\r
88 > of using mm-display-part-inline, so emacs still wasn't trying to do
\r
89 > any charset manipulation. Using mm-display-part-inline is desirable
\r
90 > because it lets us handle non-text/plain (e.g. text/html) parts
\r
91 > correctly in reply, and makes the display more consistent (since we
\r
92 > use it for show). But, it leads to this problem.
\r
94 > So, there are a couple of solutions I can see:
\r
96 > 1) Have the JSON formats include the original content-charset even
\r
97 > though they're actually outputting UTF-8. Of the solutions I tried,
\r
98 > this is the best, even though it doesn't sound like a good thing to
\r
101 > 2) Have the JSON formats include content only if it's actually UTF-8.
\r
102 > This means that for non-UTF-8 parts (including ASCII parts), the emacs
\r
103 > interface has to do more work to display the part content, since it
\r
104 > must fetch it from outside first. When I tried this, it worked but
\r
105 > caused the \nnn to show up when viewing messages in emacs. I suspect
\r
106 > this is because it sets a charset for the whole buffer, and can't
\r
107 > accommodate messages with different charsets in the same buffer
\r
108 > properly. Reply works correctly, though.
\r
110 > 3) Have the JSON formats include the charset for all parts, but make
\r
111 > it UTF-8 for all parts they include content for (since we're actually
\r
112 > outputting UTF-8). This doesn't seem to fix the problem, even though
\r
113 > it seems like it should.
\r
115 > If no one has a better idea or a strong reason not to, I'll send a
\r
116 > patch for solution (1).
\r
118 Thank you very much for your analysis. It encouraged me to dig into the
\r
119 problem and I've found another solution, which might be better than
\r
120 those you suggested.
\r
122 I traced what Emacs does with the text inside
\r
123 notmuch-mm-display-part-inline and the wrong charset conversion happens
\r
124 deeply in elisp code in mm-with-part called by mm-get-part, which is in
\r
125 turn called by mm-inline-text. There is a way to make mm-inline-text not
\r
126 to call mm-get-part, which is to set the charset to 'gnus-decoded. This
\r
127 sounds like something that applies to our situation, where the part is
\r
130 The following patch (apply it with git am -c) solves the problem for me.
\r
131 However, I'm not sure it is a universal solution. It sets the charset
\r
132 only if it is not defined in notmuch json output and I'm not sure that
\r
133 this is correct. text/html parts seem to have charset defined, but as
\r
134 you wrote that json is always utf-8, so it might be that we need
\r
135 'gnus-decoded always, independently of the json output. What do you
\r
141 diff --git a/emacs/notmuch-lib.el b/emacs/notmuch-lib.el
\r
142 index 7fa441a..8070f05 100644
\r
143 --- a/emacs/notmuch-lib.el
\r
144 +++ b/emacs/notmuch-lib.el
\r
145 @@ -244,7 +244,7 @@ the given type."
\r
146 current buffer, if possible."
\r
147 (let ((display-buffer (current-buffer)))
\r
149 - (let* ((charset (plist-get part :content-charset))
\r
150 + (let* ((charset (or (plist-get part :content-charset) 'gnus-decoded))
\r
151 (handle (mm-make-handle (current-buffer) `(,content-type (charset . ,charset)))))
\r
152 ;; If the user wants the part inlined, insert the content and
\r
153 ;; test whether we are able to inline it (which includes both
\r
projects / notmuch-archives.git / blob