Return-Path: X-Original-To: notmuch@notmuchmail.org Delivered-To: notmuch@notmuchmail.org Received: from localhost (localhost [127.0.0.1]) by olra.theworths.org (Postfix) with ESMTP id 20623431FB6 for ; Thu, 6 Dec 2012 13:15:19 -0800 (PST) X-Virus-Scanned: Debian amavisd-new at olra.theworths.org X-Spam-Flag: NO X-Spam-Score: 0 X-Spam-Level: X-Spam-Status: No, score=0 tagged_above=-999 required=5 tests=[none] autolearn=disabled Received: from olra.theworths.org ([127.0.0.1]) by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5epmUD7jef4R for ; Thu, 6 Dec 2012 13:15:18 -0800 (PST) Received: from guru.guru-group.fi (guru.guru-group.fi [46.183.73.34]) by olra.theworths.org (Postfix) with ESMTP id 2C43A431FAE for ; Thu, 6 Dec 2012 13:15:18 -0800 (PST) Received: from guru.guru-group.fi (localhost [IPv6:::1]) by guru.guru-group.fi (Postfix) with ESMTP id 625ED100045; Thu, 6 Dec 2012 23:15:13 +0200 (EET) From: Tomi Ollila To: Austin Clements , notmuch@notmuchmail.org Subject: Re: [PATCH] test: Fix UTF-8 JSON tests in Python 3 In-Reply-To: <1354634654-9564-1-git-send-email-amdragon@mit.edu> References: <87d2yp27zu.fsf@nexoid.at> <1354634654-9564-1-git-send-email-amdragon@mit.edu> User-Agent: Notmuch/0.14+116~g29fcdb5 (http://notmuchmail.org) Emacs/24.2.1 (x86_64-unknown-linux-gnu) X-Face: HhBM'cA~ MIME-Version: 1.0 Content-Type: text/plain X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Dec 2012 21:15:19 -0000 On Tue, Dec 04 2012, Austin Clements wrote: > test_expect_equal_json uses json.tool from the system Python. While > Python 2 wasn't picky about the encoding of stdin, Python 3 decodes > stdin strictly according to the environment. Since we set LC_ALL=C > for the tests, Python 3's json.tool was assuming stdin would be in > ASCII and aborting when it couldn't decode the UTF-8 characters from > some of the JSON tests. This patch sets the PYTHONIOENCODING > environment variable to utf-8 when invoking json.tool to override > Python's default encoding choice. > --- LGTM. Tomi > test/test-lib.sh | 9 +++++++-- > 1 file changed, 7 insertions(+), 2 deletions(-) > > diff --git a/test/test-lib.sh b/test/test-lib.sh > index f169785..9487526 100644 > --- a/test/test-lib.sh > +++ b/test/test-lib.sh > @@ -534,8 +534,13 @@ test_expect_equal_file () > # canonicalized before diff'ing. If an argument cannot be parsed, it > # is used unchanged so that there's something to diff against. > test_expect_equal_json () { > - output=$(echo "$1" | python -mjson.tool || echo "$1") > - expected=$(echo "$2" | python -mjson.tool || echo "$2") > + # The test suite forces LC_ALL=C, but this causes Python 3 to > + # decode stdin as ASCII. We need to read JSON in UTF-8, so > + # override Python's stdio encoding defaults. > + output=$(echo "$1" | PYTHONIOENCODING=utf-8 python -mjson.tool \ > + || echo "$1") > + expected=$(echo "$2" | PYTHONIOENCODING=utf-8 python -mjson.tool \ > + || echo "$2") > shift 2 > test_expect_equal "$output" "$expected" "$@" > } > -- > 1.7.10.4 > > _______________________________________________ > notmuch mailing list > notmuch@notmuchmail.org > http://notmuchmail.org/mailman/listinfo/notmuch