--- /dev/null
+Return-Path: <sojka@os.inf.tu-dresden.de>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+ by olra.theworths.org (Postfix) with ESMTP id 2F259431FBF\r
+ for <notmuch@notmuchmail.org>; Sun, 26 Feb 2012 01:33:21 -0800 (PST)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -2.3\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-2.3 tagged_above=-999 required=5\r
+ tests=[RCVD_IN_DNSWL_MED=-2.3] autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+ by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+ with ESMTP id vHidQBY8fa39 for <notmuch@notmuchmail.org>;\r
+ Sun, 26 Feb 2012 01:33:20 -0800 (PST)\r
+Received: from max.feld.cvut.cz (max.feld.cvut.cz [147.32.192.36])\r
+ by olra.theworths.org (Postfix) with ESMTP id 4150F431FBD\r
+ for <notmuch@notmuchmail.org>; Sun, 26 Feb 2012 01:33:20 -0800 (PST)\r
+Received: from localhost (unknown [192.168.200.4])\r
+ by max.feld.cvut.cz (Postfix) with ESMTP id F314919F33AD;\r
+ Sun, 26 Feb 2012 10:33:18 +0100 (CET)\r
+X-Virus-Scanned: IMAP AMAVIS\r
+Received: from max.feld.cvut.cz ([192.168.200.1])\r
+ by localhost (styx.feld.cvut.cz [192.168.200.4]) (amavisd-new,\r
+ port 10044)\r
+ with ESMTP id 0D5y2YkqSFhu; Sun, 26 Feb 2012 10:33:17 +0100 (CET)\r
+Received: from imap.feld.cvut.cz (imap.feld.cvut.cz [147.32.192.34])\r
+ by max.feld.cvut.cz (Postfix) with ESMTP id 99AFD19F33A6;\r
+ Sun, 26 Feb 2012 10:33:17 +0100 (CET)\r
+Received: from steelpick.2x.cz (cable-86-56-3-85.cust.telecolumbus.net\r
+ [86.56.3.85]) (Authenticated sender: sojkam1)\r
+ by imap.feld.cvut.cz (Postfix) with ESMTPSA id 60887660968;\r
+ Sun, 26 Feb 2012 10:33:17 +0100 (CET)\r
+Received: from wsh by steelpick.2x.cz with local (Exim 4.77)\r
+ (envelope-from <sojka@os.inf.tu-dresden.de>)\r
+ id 1S1aTk-00017s-EQ; Sun, 26 Feb 2012 10:33:16 +0100\r
+From: Michal Sojka <sojkam1@fel.cvut.cz>\r
+To: Serge Z <triumhiz@yandex.ru>, notmuch@notmuchmail.org\r
+Subject: Re: Double decoded text/html parts (was: [PATCH] test: Add test for\r
+ searching of uncommonly encoded messages)\r
+In-Reply-To: <20120225083600.17873.66388@localhost>\r
+References: <877gzd5axk.fsf@steelpick.2x.cz>\r
+ <1330043595-22054-1-git-send-email-sojkam1@fel.cvut.cz>\r
+ <20120224042925.2870.87924@localhost>\r
+ <874nug67il.fsf@steelpick.2x.cz>\r
+ <20120224075700.13214.28221@localhost>\r
+ <874nugiq2g.fsf@steelpick.2x.cz>\r
+ <20120225083600.17873.66388@localhost>\r
+User-Agent: Notmuch/0.11.1+239~g2e86bb7 (http://notmuchmail.org) Emacs/23.3.1\r
+ (x86_64-pc-linux-gnu)\r
+Date: Sun, 26 Feb 2012 10:33:16 +0100\r
+Message-ID: <871upihrc3.fsf@steelpick.2x.cz>\r
+MIME-Version: 1.0\r
+Content-Type: text/plain; charset=us-ascii\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+ <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Sun, 26 Feb 2012 09:33:21 -0000\r
+\r
+On Sat, 25 Feb 2012, Serge Z wrote:\r
+> \r
+> Hi!\r
+> I've struck another problem:\r
+> \r
+> I've got an html/text email with body encoded with cp1251.\r
+> Its encoding is mentioned in both Content-type: email header and html <meta>\r
+> tag. So when the client tries to display it with external html2text converter,\r
+> The message is decoded twice: first by client, second by html2text (I\r
+> use w3m).\r
+\r
+Right. After my analysis of the problem (see below) it seems there is no\r
+trivial solution for this.\r
+ \r
+> As I understand, notmuch (while indexing this message) decodes it once and\r
+> index it in the right way (though including html tags to index). But what if\r
+> the message contains no "charset" option in Content-Type email header but\r
+> contain <meta> content-type tag with charset noted?\r
+\r
+This should not happen. It violates RFC 2046, section 4.1.2.\r
+\r
+> Should such message be considered as being composed wrong or it should\r
+> be indexed with diving into html details (content-type)?\r
+\r
+I don't think it's wrongly composed and it should be even correctly\r
+indexed (with my patch). The problem is when you view such a message\r
+with an external HTML viewer.\r
+\r
+In my mailbox I can find two different types of text/html parts. First,\r
+the parts that contain complete HTML document including all headers and\r
+especially <meta http-equiv="content-type" content="text/html; ...">.\r
+Such parts could be passed to external HTML viewer without any decoding\r
+by notmuch.\r
+\r
+The second type is text/html part that does not contain any HTML\r
+headers. Passing such a part to an external HTML viewer undecoded would\r
+require it to guess the correct charset from the content.\r
+\r
+AFAIK Firefox users can set fallback charset (used for HTML documents\r
+with unknown charset) in the preferences, but I don't know what other\r
+browsers would do. In particular, do you know how w3m behaves when\r
+charset is not specified?\r
+\r
+In any way, if we want notmuch to do the right thing, we should analyze\r
+the content of text/html parts and decide whether to decode the part or\r
+not. Perhaps, a simple heuristic could be to search the content of the\r
+part for strings "charset=" and "encoding=" and if any is found, notmuch\r
+wouldn't decode that part. Otherwise it will decode it according to\r
+Content-Type header.\r
+\r
+As a curiosity, I found the following in one of my emails. Note that two\r
+different encodings (iso-8859-2 and windows-1250) are specified at the\r
+same time :) That's the reason why I think that fixing the problem won't\r
+be trivial.\r
+\r
+Content-Type: text/html; charset="iso-8859-2"\r
+Content-Transfer-Encoding: 8bit\r
+\r
+<?xml version="1.0" encoding="windows-1250" ?>\r
+<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">\r
+<html xmlns="http://www.w3.org/1999/xhtml">\r
+<head>\r
+ <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-2" />\r
+\r
+Cheers,\r
+-Michal\r
projects / notmuch-archives.git / blobdiff