From: Tomi Ollila <tomi.ollila@iki.fi>
To: David Edmondson <dme@dme.org>, Austin Clements <amdragon@MIT.EDU>,
	Pieter Praet <pieter@praet.org>
Subject: Re: [PATCH] Output unmodified Content-Type header value for JSON
	format.
In-Reply-To: <cunlip858xs.fsf@hotblack-desiato.hh.sledj.net>
References: <1321659905-24367-1-git-send-email-dmitry.kurochkin@gmail.com>
	<87fwhkyisj.fsf@servo.finestructure.net> <87wrawq1dz.fsf@gmail.com>
	<87d3coxu7s.fsf@servo.finestructure.net> <87r512pru2.fsf@gmail.com>
	<87ipmewo4z.fsf@servo.finestructure.net>
	<20111123034021.GL9351@mit.edu> <87ipkglui4.fsf@praet.org>
	<20120112172840.GC18625@mit.edu> <87ehv2proa.fsf@praet.org>
	<cun1ur16v3r.fsf@hotblack-desiato.hh.sledj.net>
	<87boq4q23z.fsf@awakening.csail.mit.edu>
	<cunlip858xs.fsf@hotblack-desiato.hh.sledj.net>
User-Agent: Notmuch/0.11+64~g42e8f66 (http://notmuchmail.org) Emacs/23.3.1
	(i686-pc-linux-gnu)
Date: Mon, 16 Jan 2012 15:18:30 +0200
Message-ID: <yf6vcobwztl.fsf@taco2.nixu.fi>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Cc: notmuch@notmuchmail.org
Precedence: list

On Mon, 16 Jan 2012 08:49:03 +0000, David Edmondson <dme@dme.org> wrote:
> On Sun, 15 Jan 2012 12:58:40 -0500, Austin Clements <amdragon@MIT.EDU> wrote:
> 
> This, I suspect, brings us back to what may have been Dmitry's original
> concern. If we codify the current behaviour then we're actually
> _forcing_ clients to use inline content if it's present, because
> otherwise they have no way to discover the charset/encoding used for the
> raw part.
> 
> That seems as though it could be a problem for some clients.
> 
> > OTOH, I don't understand the encoding story for HTML, since the encoding
> > can come from either a header or from the body of the HTML.  Does this
> > make it strictly necessary for the client to handle the encoding?

Either from a header or from the body of the *HTTP*. Html meta tag is part
of the header of the HTML document, body of the HTTP message.

> If it can be specified by the content of a part rather than the part
> headers, then I think that the client will have to be prepared to handle
> it.
> 
> Even if not, it might still be more effective to choose that approach -
> it would remove the need to add arbitrary encoding support to the CLI
> application.

In case of w3m interface if charset is not told it defaults
to iso-8859-1 as both input and output encoding.

That is no problem in w3m -- it just doesn't do any conversion.

But emacs thinks that it gets input in iso-8859-1 format and will
attempt to convert that to whatever charset the window user has
is using. 

If input was utf8 but emacs thinks input was latin1 then we get
this infamous 'double-utf8'ied output (a subset of wtf-8 charset ;)
(in case window charset is utf8)

In case of w3m the content feed to w3m could be pre-encoded to utf-8
and w3m interface is told that charset is utf-8 -- w3m will now
called using utf-8 as input and output encoding -- and at the end
emacs does conversion from utf-8 to the window encoding (if needed).

As mentioned in IRC: 2012-01-10 11:46 (UTC)  xxxXX  indeed, the headers
	should take precedence to meta tag, as defined in HTML4

So, complying clients takes the precedence from command line options
(which, in case of command line clients makes perfect sense).

Tomi