Re: [PATCH v1 1/2] emacs: Observe the charset of MIME parts when reading them.

diff --git a/6a/aea24b9a4e6354a810106b483e9d9bca3a48ff b/6a/aea24b9a4e6354a810106b483e9d9bca3a48ff
new file mode 100644 (file)
index 0000000..e27240f
--- /dev/null
+++ b/6a/aea24b9a4e6354a810106b483e9d9bca3a48ff
@@ -0,0 +1,103 @@
+Return-Path: <dme@dme.org>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+ by arlo.cworth.org (Postfix) with ESMTP id 92A086DE00F5\r
+ for <notmuch@notmuchmail.org>; Tue,  3 May 2016 03:28:28 -0700 (PDT)\r
+X-Virus-Scanned: Debian amavisd-new at cworth.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -0.185\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-0.185 tagged_above=-999 required=5\r
+ tests=[AWL=-0.118, DKIM_SIGNED=0.1, DKIM_VALID=-0.1,\r
+ RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01,\r
+ RCVD_IN_MSPIKE_WL=-0.01, SPF_NEUTRAL=0.652, UNPARSEABLE_RELAY=0.001]\r
+ autolearn=disabled\r
+Received: from arlo.cworth.org ([127.0.0.1])\r
+ by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024)\r
+ with ESMTP id E_ghINMwMZbU for <notmuch@notmuchmail.org>;\r
+ Tue,  3 May 2016 03:28:19 -0700 (PDT)\r
+Received: from mail-wm0-f52.google.com (mail-wm0-f52.google.com\r
+ [74.125.82.52]) by arlo.cworth.org (Postfix) with ESMTPS id 679A46DE00CE for\r
+ <notmuch@notmuchmail.org>; Tue,  3 May 2016 03:28:19 -0700 (PDT)\r
+Received: by mail-wm0-f52.google.com with SMTP id a17so30561623wme.0\r
+ for <notmuch@notmuchmail.org>; Tue, 03 May 2016 03:28:19 -0700 (PDT)\r
+DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;\r
+ d=dme-org.20150623.gappssmtp.com; s=20150623;\r
+ h=to:subject:in-reply-to:references:from:date:message-id:mime-version;\r
+ bh=44rp5rV/jE5sc0G3Orm4ePHyRcZ8a+kJe4dXrucmKtU=;\r
+ b=axOuh4otVcojPFkDgQfJmAGEsceNAQiiwyo1EkAxVj5N4hegclWStM4HnRYYlDaToq\r
+ M64IBEkvVvChWXNe5xtfsMBwndwCLjEzxsYYbwXZKIomQw2XtO+u1e0xjAwdBn0Xzt9z\r
+ khFICzXrl4zZIhpdUt80hP3AmiXbKxKQfZCdQFlj0GT31hU0MKoZoEovKa6z/eM86A1E\r
+ hKTwrEg+7tbsLQyNEyehexWS8HtxNBnZykAB4pvohGw2bBgsShF3ETBrHwmjYWt6Ffj7\r
+ SsN8adYZr8T/mgOsM8YLJHFKAPBgdxcpOuMEaA0NUEx3FBUKWhlkC4Na1g15TFb7jsa9\r
+ zd7g==\r
+X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;\r
+ d=1e100.net; s=20130820;\r
+ h=x-gm-message-state:to:subject:in-reply-to:references:from:date\r
+ :message-id:mime-version;\r
+ bh=44rp5rV/jE5sc0G3Orm4ePHyRcZ8a+kJe4dXrucmKtU=;\r
+ b=mRR1kC7evG/0MbybZWxVt4LsqXNpfBN+r3XjMzk8yrcqpbioh5KfTZYw5UP/6jhxcr\r
+ 3bGzNZfg0SqXkwML5YvZvaW+xtcGYOpNB1P6rStZyZuYhQa2Bbq030o5kZtoHFSoSM8V\r
+ QjEfNkBLOBeV3iqjpFzv9j9GlL2W88JgcHyGuk/0tMkpk53PKvmvPjmegNLSsyEmL6Zk\r
+ T8zYraiAMm2CnvMR8Z76jI3yYJBCpxwRHS6jGaz8ahLE24ZO5qFnBhcVQGuWgxi0txo1\r
+ Et++sYoT6VM+520c3B6jAT3xQfO+phdCgHh2c6XbfBU3HTSwN33sZyVcOR4V6K03+9yQ\r
+ 5azQ==\r
+X-Gm-Message-State:\r
+ AOPr4FVOVEOhRK7WeWBuMt8+rolVNhM9ryyRQ1Wd+RBvGkRiLprPyMAKYoajWnU8SO5k+Q==\r
+X-Received: by 10.28.6.138 with SMTP id 132mr23270653wmg.60.1462271298049;\r
+ Tue, 03 May 2016 03:28:18 -0700 (PDT)\r
+Received: from disaster-area.hh.sledj.net\r
+ ([2a01:348:1a2:1:ea39:35ff:fe2c:a227])\r
+ by smtp.gmail.com with ESMTPSA id d79sm23751800wmi.23.2016.05.03.03.28.16\r
+ (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);\r
+ Tue, 03 May 2016 03:28:16 -0700 (PDT)\r
+Received: from localhost (disaster-area.hh.sledj.net [local])\r
+ by disaster-area.hh.sledj.net (OpenSMTPD) with ESMTPA id 87fe421f;\r
+ Tue, 3 May 2016 10:28:15 +0000 (UTC)\r
+To: Mark Walters <markwalters1009@gmail.com>, notmuch@notmuchmail.org\r
+Subject: Re: [PATCH v1 1/2] emacs: Observe the charset of MIME parts when\r
+ reading them.\r
+In-Reply-To: <877ffc9agl.fsf@qmul.ac.uk>\r
+References: <1461999108-68582-1-git-send-email-dme@dme.org>\r
+ <1461999108-68582-2-git-send-email-dme@dme.org> <877ffc9agl.fsf@qmul.ac.uk>\r
+From: David Edmondson <dme@dme.org>\r
+Date: Tue, 03 May 2016 11:28:15 +0100\r
+Message-ID: <m2a8k7e8qo.fsf@dme.org>\r
+MIME-Version: 1.0\r
+Content-Type: text/plain\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.20\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+ <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <https://notmuchmail.org/mailman/options/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch/>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <https://notmuchmail.org/mailman/listinfo/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Tue, 03 May 2016 10:28:28 -0000\r
+\r
+On Mon, May 02 2016, Mark Walters wrote:\r
+> On Sat, 30 Apr 2016, David Edmondson <dme@dme.org> wrote:\r
+>> `notmuch--get-bodypart-raw' previously assumed that all non-binary MIME\r
+>> parts could be successfully read by assuming that they were UTF-8\r
+>> encoded. This was demonstrated to be wrong, specifically when a part was\r
+>> marked as ISO8859-1 and included accented characters (which were\r
+>> incorrectly rendered as a result).\r
+>>\r
+>> Rather than assuming UTF-8, attempt to use the part's declared charset\r
+>> when reading it, falling back to US-ASCII if the declared charset is\r
+>> unknown, unsupported or invalid.\r
+>\r
+> As this seemed hard to test (if I understand the bug correctly it didn't\r
+> show up on my test of the entire of the entire performance corpus -- of\r
+> course my testing could have been wrong) would it be possible to add a test\r
+> for it?\r
+\r
+I agree that we should have a test, and I will make one.\r
+\r
+(/me lives in fear of the day when bremner introduces a "no fixes\r
+without tests" policy...)\r