From: Dmitry Kurochkin <dmitry.kurochkin@gmail.com>
To: Austin Clements <amdragon@MIT.EDU>
Subject: Re: [PATCH] Output unmodified Content-Type header value for JSON
	format.
In-Reply-To: <20111119185818.GR9351@mit.edu>
References: <1321659905-24367-1-git-send-email-dmitry.kurochkin@gmail.com>
	<87fwhkyisj.fsf@servo.finestructure.net>
	<87wrawq1dz.fsf@gmail.com> <20111119045957.GQ9351@mit.edu>
	<87ty60pts9.fsf@gmail.com> <20111119185818.GR9351@mit.edu>
User-Agent: Notmuch/0.10~rc1+20~gec94ced (http://notmuchmail.org) Emacs/23.3.1
	(x86_64-pc-linux-gnu)
Date: Sun, 20 Nov 2011 22:52:39 +0400
Message-ID: <87lirapqx4.fsf@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Cc: notmuch@notmuchmail.org
Precedence: list

On Sat, 19 Nov 2011 13:58:18 -0500, Austin Clements <amdragon@MIT.EDU> wrote:
> Quoth Dmitry Kurochkin on Nov 19 at  9:26 am:
> > On Fri, 18 Nov 2011 23:59:57 -0500, Austin Clements <amdragon@MIT.EDU> wrote:
> > > Quoth Dmitry Kurochkin on Nov 19 at  6:42 am:
> > > > Hi Jamie.
> > > > 
> > > > On Fri, 18 Nov 2011 17:58:52 -0800, Jameson Graef Rollins <jrollins@finestructure.net> wrote:
> > > > > On Sat, 19 Nov 2011 03:45:05 +0400, Dmitry Kurochkin <dmitry.kurochkin@gmail.com> wrote:
> > > > > > Before the change, notmuch used g_mime_content_type_to_string(3)
> > > > > > function to output Content-Type header value.  Turns out it outputs
> > > > > > only "type/subtype" part and ignores all parameters.  Also, if there
> > > > > > is no Content-Type header, default "text/plain" value is used.
> > > > > 
> > > > > Hi, Dmitry.  Can you explain under what circumstances you would need the
> > > > > extra content-type parameters?
> > > > 
> > > > Charset is an example of a parameter which you need to render a part
> > > > correctly.
> > > 
> > > Can notmuch convert to a common charset, given that, otherwise, every
> > > client is going to have to implement this conversion anyway?
> > > 
> > 
> > Notmuch can handle charset (and any other) parameters but only for known
> > media types (i.e. text/*).  I think it would be useful (especially for
> > human-readable output formats).  But it is a separate issue.
> > 
> > Notmuch can not convert other types it does not know how to handle.
> > E.g. HTML charset conversion is not as simple as for plain text.
> > 
> > AFAIK standard defines charset parameter just for few types.  So in
> > general, charset parameter can have any meaning for some custom media
> > type.
> 
> Interesting.  I hadn't realized the content-type specification was so
> open-ended.  However, there are many things that *could* be included
> in the JSON format but aren't; what's included is primarily driven by
> what consumers actually need and it seems like the actual need here is
> charset handling.  Maybe the JSON format *shouldn't* evolve this way,
> but I think it should either be driven by its needs like it is now, or
> we should be taking bigger steps like providing *all* of the headers
> (essentially, a JSON-ification of the MIME structure), which would
> subsume more specific generalizations like exposing just the full
> content-type header.
> 

I think it is a good idea to provide all headers in JSON output.  Still
I believe this patch is still valid.  It is a simple change, which makes
the JSON format simpler and we have consumers that need it.  Providing
all headers would be a bigger change (and I expect it to be much more
difficult to get accepted).

What I definately do not like, is adding an exception for charset
parameter and inventing complex rules for JSON format instead of keeping
it simple.

> Regarding charset, specifically, though, the JSON format only includes
> part bodies for text/* types and, according to RFC 2045,
> 
>   For example, the "charset" parameter is applicable to any subtype of
>   "text", ...
> 
> Section 4.1.2 (Charset Parameter) of RFC 2046 beats around the bush,
> but I think it's saying essentially the same thing in a lot more
> detail.  Given that, I think it does make sense for notmuch to handle
> the charset parameter and re-coding.
> 

I think it may be a good idea but it is not trivial to do right.  We
should not just convert all text parts unconditionally to locale or
UTF-8.

> > > (And are there other examples of useful things in the content type?)
> > 
> > What is meant by useful?  All parameters do have some use.  The fact
> > that notmuch does not handle them does not mean they are useless.  And
> > notmuch can not handle all parameters just because the list of
> > parameters is not defined.  So there is no choice but to let notmuch
> > users see and use these parameters.
> 
> Yes, I now agree with this, modulo my statements about generality above.
> 

Thanks.

Regards,
  Dmitry

> > Regards,
> >   Dmitry
> > 
> 
> -- 
> Austin Clements                                      MIT/'06/PhD/CSAIL
> amdragon@mit.edu                           http://web.mit.edu/amdragon
>        Somewhere in the dream we call reality you will find me,
>               searching for the reality we call dreams.