[PATCH] test: Fix UTF-8 JSON tests in Python 3

diff --git a/c7/b2b7c8403e7ba56feda75fb9eeb7c742adba0b b/c7/b2b7c8403e7ba56feda75fb9eeb7c742adba0b
new file mode 100644 (file)
index 0000000..3750da4
--- /dev/null
+++ b/c7/b2b7c8403e7ba56feda75fb9eeb7c742adba0b
@@ -0,0 +1,107 @@
+Return-Path: <amdragon@mit.edu>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+       by olra.theworths.org (Postfix) with ESMTP id 6C48D431FB6\r
+       for <notmuch@notmuchmail.org>; Tue,  4 Dec 2012 07:24:25 -0800 (PST)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -0.7\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-0.7 tagged_above=-999 required=5\r
+       tests=[RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+       by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+       with ESMTP id tUpnjUcjgZ0e for <notmuch@notmuchmail.org>;\r
+       Tue,  4 Dec 2012 07:24:21 -0800 (PST)\r
+Received: from dmz-mailsec-scanner-3.mit.edu (DMZ-MAILSEC-SCANNER-3.MIT.EDU\r
+       [18.9.25.14])\r
+       by olra.theworths.org (Postfix) with ESMTP id 76DE1431FAE\r
+       for <notmuch@notmuchmail.org>; Tue,  4 Dec 2012 07:24:21 -0800 (PST)\r
+X-AuditID: 1209190e-b7f516d0000008e4-f5-50be15a43521\r
+Received: from mailhub-auth-1.mit.edu ( [18.9.21.35])\r
+       by dmz-mailsec-scanner-3.mit.edu (Symantec Messaging Gateway) with SMTP\r
+       id 23.95.02276.4A51EB05; Tue,  4 Dec 2012 10:24:20 -0500 (EST)\r
+Received: from outgoing.mit.edu (OUTGOING-AUTH.MIT.EDU [18.7.22.103])\r
+       by mailhub-auth-1.mit.edu (8.13.8/8.9.2) with ESMTP id qB4FOJN5009216; \r
+       Tue, 4 Dec 2012 10:24:19 -0500\r
+Received: from drake.dyndns.org\r
+       (209-6-116-242.c3-0.arl-ubr1.sbo-arl.ma.cable.rcn.com\r
+       [209.6.116.242]) (authenticated bits=0)\r
+       (User authenticated as amdragon@ATHENA.MIT.EDU)\r
+       by outgoing.mit.edu (8.13.6/8.12.4) with ESMTP id qB4FOG6A012417\r
+       (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NOT);\r
+       Tue, 4 Dec 2012 10:24:18 -0500 (EST)\r
+Received: from amthrax by drake.dyndns.org with local (Exim 4.77)\r
+       (envelope-from <amdragon@mit.edu>)\r
+       id 1TfuM4-0002Ux-Iv; Tue, 04 Dec 2012 10:24:16 -0500\r
+From: Austin Clements <amdragon@MIT.EDU>\r
+To: notmuch@notmuchmail.org\r
+Subject: [PATCH] test: Fix UTF-8 JSON tests in Python 3\r
+Date: Tue,  4 Dec 2012 10:24:14 -0500\r
+Message-Id: <1354634654-9564-1-git-send-email-amdragon@mit.edu>\r
+X-Mailer: git-send-email 1.7.10.4\r
+In-Reply-To: <87d2yp27zu.fsf@nexoid.at>\r
+References: <87d2yp27zu.fsf@nexoid.at>\r
+X-Brightmail-Tracker:\r
+ H4sIAAAAAAAAA+NgFtrDIsWRmVeSWpSXmKPExsUixCmqrLtEdF+AwZ5fNhZ7G9oZLa7fnMns\r
+       wOSxeNN+No9nq24xBzBFcdmkpOZklqUW6dslcGVMf/iUseAvd8WV3weZGhiPcnYxcnJICJhI\r
+       7Duzkg3CFpO4cG89kM3FISSwj1Fi+rezrBDOekaJ/mV/mCCch0wSy1pvgbUICcxllHjzWA/E\r
+       ZhPQkNi2fzkjiC0iIC2x8+5sVhCbWcBMYu7PiWD1wgLmEnM7J4HVsAioSrxZf5ili5GDg1fA\r
+       XuLCLluIKxQlup9NACvnFFCXWPhjHiPEKjWJaSces05g5F/AyLCKUTYlt0o3NzEzpzg1Wbc4\r
+       OTEvL7VI11gvN7NELzWldBMjOJAk+XYwfj2odIhRgINRiYd3wq09AUKsiWXFlbmHGCU5mJRE\r
+       eY2F9wUI8SXlp1RmJBZnxBeV5qQWH2KU4GBWEuFV+LA3QIg3JbGyKrUoHyYlzcGiJM57JeWm\r
+       v5BAemJJanZqakFqEUxWhoNDSYJ3kQjQUMGi1PTUirTMnBKENBMHJ8hwHqDhWSA1vMUFibnF\r
+       mekQ+VOMilLivKwgCQGQREZpHlwvLNJfMYoDvSLMew3kbh5gkoDrfgU0mAlo8Auh3SCDSxIR\r
+       UlINjB3bLylOPxMoMt94/czWZ9LcLNpdJ67/tAwwkE26tvW+3KfEvoLYTTu+iR2YbvP+A+O2\r
+       jheBB4U4/81QdO5et+Zd4A/5P3pBVzJ+lcRk2+hxTXQ0ZLwlFt4sND/hg63Mx23fdzmw3/lq\r
+       EW/4dlJp1d3TLpa7XU6sPfgg8VTpl8AVqszqwe8bYpRYijMSDbWYi4oTAXKN1AnPAgAA\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+       <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Tue, 04 Dec 2012 15:24:25 -0000\r
+\r
+test_expect_equal_json uses json.tool from the system Python.  While\r
+Python 2 wasn't picky about the encoding of stdin, Python 3 decodes\r
+stdin strictly according to the environment.  Since we set LC_ALL=C\r
+for the tests, Python 3's json.tool was assuming stdin would be in\r
+ASCII and aborting when it couldn't decode the UTF-8 characters from\r
+some of the JSON tests.  This patch sets the PYTHONIOENCODING\r
+environment variable to utf-8 when invoking json.tool to override\r
+Python's default encoding choice.\r
+---\r
+ test/test-lib.sh |    9 +++++++--\r
+ 1 file changed, 7 insertions(+), 2 deletions(-)\r
+\r
+diff --git a/test/test-lib.sh b/test/test-lib.sh\r
+index f169785..9487526 100644\r
+--- a/test/test-lib.sh\r
++++ b/test/test-lib.sh\r
+@@ -534,8 +534,13 @@ test_expect_equal_file ()\r
+ # canonicalized before diff'ing.  If an argument cannot be parsed, it\r
+ # is used unchanged so that there's something to diff against.\r
+ test_expect_equal_json () {\r
+-    output=$(echo "$1" | python -mjson.tool || echo "$1")\r
+-    expected=$(echo "$2" | python -mjson.tool || echo "$2")\r
++    # The test suite forces LC_ALL=C, but this causes Python 3 to\r
++    # decode stdin as ASCII.  We need to read JSON in UTF-8, so\r
++    # override Python's stdio encoding defaults.\r
++    output=$(echo "$1" | PYTHONIOENCODING=utf-8 python -mjson.tool \\r
++        || echo "$1")\r
++    expected=$(echo "$2" | PYTHONIOENCODING=utf-8 python -mjson.tool \\r
++        || echo "$2")\r
+     shift 2\r
+     test_expect_equal "$output" "$expected" "$@"\r
+ }\r
+-- \r
+1.7.10.4\r
+\r