From: W. Trevor King Date: Tue, 4 Feb 2014 19:14:53 +0000 (+1600) Subject: Re: [PATCH 00/17] nmbug-status: Python-3-compabitility and general refactoring X-Git-Url: http://git.tremily.us/?a=commitdiff_plain;h=0726f9b519d309c2c51f9ce0bf035daa115dd4fd;p=notmuch-archives.git Re: [PATCH 00/17] nmbug-status: Python-3-compabitility and general refactoring --- diff --git a/6b/0fd4053a758faa7177a3ea7a761f3b9de997ec b/6b/0fd4053a758faa7177a3ea7a761f3b9de997ec new file mode 100644 index 000000000..31b0476d1 --- /dev/null +++ b/6b/0fd4053a758faa7177a3ea7a761f3b9de997ec @@ -0,0 +1,195 @@ +Return-Path: +X-Original-To: notmuch@notmuchmail.org +Delivered-To: notmuch@notmuchmail.org +Received: from localhost (localhost [127.0.0.1]) + by olra.theworths.org (Postfix) with ESMTP id 511DD431FBD + for ; Tue, 4 Feb 2014 11:15:02 -0800 (PST) +X-Virus-Scanned: Debian amavisd-new at olra.theworths.org +X-Spam-Flag: NO +X-Spam-Score: -0.1 +X-Spam-Level: +X-Spam-Status: No, score=-0.1 tagged_above=-999 required=5 + tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, + RCVD_IN_DNSWL_NONE=-0.0001] autolearn=disabled +Received: from olra.theworths.org ([127.0.0.1]) + by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) + with ESMTP id JO79NcFFFMcn for ; + Tue, 4 Feb 2014 11:14:56 -0800 (PST) +Received: from qmta07.westchester.pa.mail.comcast.net + (qmta07.westchester.pa.mail.comcast.net [76.96.62.64]) + by olra.theworths.org (Postfix) with ESMTP id 3B56C431FBC + for ; Tue, 4 Feb 2014 11:14:56 -0800 (PST) +Received: from omta15.westchester.pa.mail.comcast.net ([76.96.62.87]) + by qmta07.westchester.pa.mail.comcast.net with comcast + id NEvm1n0061swQuc57KEv6T; Tue, 04 Feb 2014 19:14:55 +0000 +Received: from odin.tremily.us ([24.18.63.50]) + by omta15.westchester.pa.mail.comcast.net with comcast + id NKEu1n00D152l3L3bKEu7Q; Tue, 04 Feb 2014 19:14:55 +0000 +Received: by odin.tremily.us (Postfix, from userid 1000) + id A69BEFCF3BF; Tue, 4 Feb 2014 11:14:53 -0800 (PST) +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tremily.us; s=odin; + t=1391541293; bh=teo73fKl9lcjS3Vb2MBIhj/ic7aYnmhEU1fGWCm2BGg=; + h=Date:From:To:Cc:Subject:References:In-Reply-To; + b=lDdSoAXBH65QYVoaYdsIY4h8p3TIBodFnkyNh/ewgXW0DOQSMTTzrrgI/3o/Mglhc + +p9qwKxiDKlPD7cdwB7MOcqHij4PmVmiMCLAL/CL+nRoCEdc7ADviF90gRygCHNo4n + mwMhJHlQBDVSBINBRzSScO45vlNVups29t9W+sJ4= +Date: Tue, 4 Feb 2014 11:14:53 -0800 +From: "W. Trevor King" +To: Tomi Ollila +Subject: Re: [PATCH 00/17] nmbug-status: Python-3-compabitility and general + refactoring +Message-ID: <20140204191453.GV14197@odin.tremily.us> +References: + + <20140204005331.GQ14197@odin.tremily.us> + + <20140204161142.GS14197@odin.tremily.us> + +MIME-Version: 1.0 +Content-Type: multipart/signed; micalg=pgp-sha1; + protocol="application/pgp-signature"; boundary="zD4kWSsXNMryRTZe" +Content-Disposition: inline +In-Reply-To: +OpenPGP: id=39A2F3FA2AB17E5D8764F388FC29BDCDF15F5BE8; + url=http://tremily.us/pubkey.txt +User-Agent: Mutt/1.5.22 (2013-10-16) +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=comcast.net; + s=q20121106; t=1391541295; + bh=qgEIttGAOUL7mjviI7bfVbSGUdx/aC+Ofh/r1bXhBJs=; + h=Received:Received:Received:Date:From:To:Subject:Message-ID: + MIME-Version:Content-Type; + b=R8dEWjoQHwgIcEpoJSOuN+lQTqSHTVJOXDYgbZD4p3exkhQuspZWIkyJhkxizGN9C + Fzodb1F9jdGDIh36XqmazhcgoSzW7G167dqq6vrHxfOkL1AzI2ml/H1SjGe0yASoCE + oMNa3iNH/JdvTQ3lIzJSixUmZB+zKbIorOXgFJzG1Oy2doxO/KGzQm9GJvWTqjTqyN + PSMUp8/Uv2DJSJoV9zJFkqMirP1uqeq5wz2BK0DWPzDbEHoNS5rM8AZ05YNJYgco5h + hBxDzXM7BPXh8b421KjCt1pL9TTZ0Af763fnI6bf5D8lw+FftJXAjydn0HhO8BuPsQ + CU+WdeTDTcffg== +Cc: notmuch@notmuchmail.org +X-BeenThere: notmuch@notmuchmail.org +X-Mailman-Version: 2.1.13 +Precedence: list +List-Id: "Use and development of the notmuch mail system." + +List-Unsubscribe: , + +List-Archive: +List-Post: +List-Help: +List-Subscribe: , + +X-List-Received-Date: Tue, 04 Feb 2014 19:15:02 -0000 + + +--zD4kWSsXNMryRTZe +Content-Type: text/plain; charset=utf-8 +Content-Disposition: inline +Content-Transfer-Encoding: quoted-printable + +On Tue, Feb 04, 2014 at 08:40:18PM +0200, Tomi Ollila wrote: +> On Tue, Feb 04 2014, W. Trevor King wrote: +> > +> > >>> from __future__ import unicode_literals +> > >>> import codecs +> > >>> import locale +> > >>> import sys +> > >>> print(locale.getpreferredencoding()) # same as yours +> > UTF-8 +> > >>> print(sys.getdefaultencoding()) # same as yours +> > ascii +> > >>> _ENCODING =3D locale.getpreferredencoding() or sys.getdefaultenco= +ding() +> > >>> print(_ENCODING) # double-check default encodings +> > UTF-8 +> > >>> byte_stream =3D sys.stdout # copied from Page.write +> > >>> stream =3D codecs.getwriter(encoding=3D_ENCODING)(stream=3Dbyte_s= +tream) +> > >>> data =3D {'from': '\u017b'} # fake the troublesome data +> > >>> print(type(data['from'])) # double-check unicode_literals +> > +> > >>> string =3D ' {from}\n'.format(**data) +> > >>> stream.write(string) +> > =C5=BB +> > +> > It looks like you'll have the same _ENCODING as I do (UTF-8). That +> > means your stream should be wrapped in a UTF-8 StreamWriter, so I +> > don't understand why it's converting to ASCII. Can you run through +> > the above on your troublesome machine and confirm that stream.write() +> > is still raising the exception? If it doesn't work, can you just +> > paste that whole run in your next email? +>=20 +> I don't know what to paste, so i paste this: +>=20 +> $ python +> Python 2.6.6 (r266:84292, Nov 21 2013, 12:39:37)=20 +> [GCC 4.4.7 20120313 (Red Hat 4.4.7-3)] on linux2 +> Type "help", "copyright", "credits" or "license" for more information. + +It looks like you left out: + + from __future__ import unicode_literals + +Can you try again with that line as the first command? + +> >>> data =3D {'from': '\u017b'} +> >>> print(type(data['from']))=20 +> + +which is why your data is a 'str' and not a 'unicode' instance. + +> >>> string =3D ' {from}\n'.format(**data) +> >>> print string +> \u017b +>=20 +> and then: +>=20 +> >>> data =3D {'from': u'\u017b'} + +This works around the lack of unicode_literals with an explicit u''. + +> >>> print(type(data['from']))=20 +> +> >>> string =3D ' {from}\n'.format(**data) +> Traceback (most recent call last): +> File "", line 1, in +> UnicodeEncodeError: 'ascii' codec can't encode character u'\u017b' in + +However, without unicode_literals or an explicit u'', you're format +string '=E2=80=A6{from}' is a str (it should be a 'unicode' instance with +unicode_literals). + +> >>> import os +> >>> print os.environ['LANG'] +> en_US.UTF-8 + +That's good anyway ;). Thanks for digging into this :). + +Cheers, +Trevor + +--=20 +This email may be signed or encrypted with GnuPG (http://www.gnupg.org). +For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy + +--zD4kWSsXNMryRTZe +Content-Type: application/pgp-signature; name="signature.asc" +Content-Description: OpenPGP digital signature + +-----BEGIN PGP SIGNATURE----- +Version: GnuPG v2.0.22 (GNU/Linux) + +iQIcBAEBAgAGBQJS8TwsAAoJEKKfehoaNkbtRPAP/i/PtIT6gMW14hkLuPy1BksK +m7TuPzNokEAE0+RVQF2OIFrK3VboowMgQZICL70ZUIYZmp3vSyJv4m9SWH2N2gw9 +0USZhga1OYUmS3eUucFjfuCK5dVXYOxUtRvXoUZh3AgHjV0V6bIu0QpTCJghEKSG +Z5k4Bway4dV/CjNFQsHFzqpsw3W93+eOWQdi6ddV0v2yzSc7h8eVlB/i2DHcvSXX +VjHwPnKVt8UnEWUf9MyYGrSDzUfJrmr60ux4ntTykHNvlDV7gX2I1yHtx/K/iwks +n31fARQmM3F7r3sAkM4IWHT/+bjRfD+k5S2EnQDdNadR1T0etO/+ybrFZwTB8nmV +gwdHdOylsJcqiFl9y2tbplVTdxnAKy8y1di2zxYCEGNL4BhWv2ak8wFz9dMkAs9W +hUI6Q0GcCR0E9BWksts6FJYP+ZgoFv4tKrozW++5VVy0gExQPv8bF4wOngCnibqk +Ir7Ec6K88N4Az320xdTzQZR+nr9i272f5Aa4XOOFfEJpZQrH1eIe5Eiu0jymraen +SybaBMGDlAy4D4c7FXJ6XlPWolUH1nw8fVq3eoDq5qN5/b6F05cb8ouP1vG2q8fn +dg2QllsAD6GB001nzxCMo5JlwBdTw/l78f2aujeK7L6toWSWWBfgKNfddZWq6CKu +o52ndagLxMGPpKoY15og +=gPZZ +-----END PGP SIGNATURE----- + +--zD4kWSsXNMryRTZe--