--- /dev/null
+Return-Path: <wking@tremily.us>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+ by arlo.cworth.org (Postfix) with ESMTP id 181856DE0A87\r
+ for <notmuch@notmuchmail.org>; Sun, 14 Feb 2016 21:29:11 -0800 (PST)\r
+X-Virus-Scanned: Debian amavisd-new at cworth.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: 0.057\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=0.057 tagged_above=-999 required=5 tests=[AWL=0.058, \r
+ DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001,\r
+ SPF_PASS=-0.001] autolearn=disabled\r
+Received: from arlo.cworth.org ([127.0.0.1])\r
+ by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024)\r
+ with ESMTP id YwGviKfM47oN for <notmuch@notmuchmail.org>;\r
+ Sun, 14 Feb 2016 21:29:08 -0800 (PST)\r
+Received: from resqmta-po-08v.sys.comcast.net (resqmta-po-08v.sys.comcast.net\r
+ [96.114.154.167])\r
+ by arlo.cworth.org (Postfix) with ESMTPS id F3AA26DE0943\r
+ for <notmuch@notmuchmail.org>; Sun, 14 Feb 2016 21:29:07 -0800 (PST)\r
+Received: from resomta-po-09v.sys.comcast.net ([96.114.154.233])\r
+ by resqmta-po-08v.sys.comcast.net with comcast\r
+ id JVUu1s00552QWKC01VV5xF; Mon, 15 Feb 2016 05:29:05 +0000\r
+Received: from mail.tremily.us ([73.221.72.168])\r
+ by resomta-po-09v.sys.comcast.net with comcast\r
+ id JVV31s00K3dr3C901VV4g9; Mon, 15 Feb 2016 05:29:05 +0000\r
+Received: from ullr.tremily.us (unknown [192.168.10.7])\r
+ by mail.tremily.us (Postfix) with ESMTPS id 13BA61BB2A0D;\r
+ Sun, 14 Feb 2016 21:29:03 -0800 (PST)\r
+Received: (nullmailer pid 22232 invoked by uid 1000);\r
+ Mon, 15 Feb 2016 05:30:14 -0000\r
+From: "W. Trevor King" <wking@tremily.us>\r
+To: notmuch@notmuchmail.org\r
+Subject: [PATCH] nmbug: Allow Unicode tags and IDs in Python 2\r
+Date: Sun, 14 Feb 2016 21:30:11 -0800\r
+Message-Id:\r
+ <e287050a10ce1d2120db996d2d200f610370a44e.1455513965.git.wking@tremily.us>\r
+X-Mailer: git-send-email 2.1.0.60.g85f0837\r
+DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=comcast.net;\r
+ s=q20140121; t=1455514145;\r
+ bh=JSA6UzUGX/hI8zjlpzXe0gXS06y41p1jTP778uTzK/M=;\r
+ h=Received:Received:Received:Received:From:To:Subject:Date:\r
+ Message-Id;\r
+ b=p/x4GLjW56OLBfGD4zuYDpy+VrFGH6arhZpb4jqVbHWS/O3bUYtwg6uoe0oX2nkDg\r
+ 1iTNLUrAR6WJWCiTevS4evair0kNyvidoqEwc6kbLn1+hc2qaCB6khcDbDixrIt2UC\r
+ ihgTMKaxqvR/HppE0jLc79gLN2HR7fKHy3RbSGDyAlJx5+wRiUi9hq+Lof9Qf5adkT\r
+ ZJNqiUnXtF8HXGjbLSTkX0VemiBYLGHh2meLN+hCcKPXCbFh47XkZXuL/uIzrmoJ2y\r
+ T9BrsWTry9Pmk4v6VMv/SgtG9z40IhLhwNIgfMI9OQlubaXg5oTY77xqHYbqVZpX0I\r
+ gwwPCDf0XO5Bg==\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.20\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+ <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <https://notmuchmail.org/mailman/options/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch/>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <https://notmuchmail.org/mailman/listinfo/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Mon, 15 Feb 2016 05:29:11 -0000\r
+\r
+Avoid a UnicodeWarning and broken pipe on 'nmbug commit' in Python 2\r
+when a tag or message ID contains non-ASCII characters [1].\r
+\r
+There are a number of Python bugs associated with this behavior\r
+[2,3,4,5,6]. There's also some useful background in [8]. [3] lead to\r
+the currently working Python 3 implementation, which encodes to UTF-8\r
+by default and has 'encoding' and 'errors' arguments [7]. This commit\r
+follows that approach in a way that's compatible with both Python 2\r
+and Python 3. Coercing to UTF-8 (regardless of locale) gives us\r
+consistent tag IDs for sharing between users.\r
+\r
+The 'isnumeric' check identifies Unicode instances in both Python 2\r
+[9] and Python 3 [10].\r
+\r
+[1]: id:87twlbv5vj.fsf@zancas.localnet\r
+ http://thread.gmane.org/gmane.mail.notmuch.general/21855/focus=21862\r
+ Subject: Re: problems with nmbug and empty prefix (UnicodeWarning and broken pipe)\r
+ Date: Sun, 14 Feb 2016 08:22:24 -0400\r
+[2]: http://bugs.python.org/issue2637\r
+[3]: http://bugs.python.org/issue3300\r
+[4]: http://bugs.python.org/issue22231\r
+[5]: http://bugs.python.org/issue23885\r
+[6]: http://bugs.python.org/issue1712522\r
+[7]: https://docs.python.org/3/library/urllib.parse.html#urllib.parse.quote\r
+[8]: https://mail.python.org/pipermail/python-dev/2006-July/067335.html\r
+[9]: https://docs.python.org/2/library/stdtypes.html#unicode.isnumeric\r
+[10]: https://docs.python.org/3/library/stdtypes.html#str.isnumeric\r
+---\r
+I haven't checked the other commands for issues with Unicode IDs or\r
+tags. It's possible that in addition to this explicit encoding to\r
+UTF-8, we'll also want explicit decoding from UTF-8 when reading from\r
+Git trees (for 'nmbug checkout' and 'nmbug status').\r
+\r
+Cheers,\r
+Trevor\r
+\r
+ devel/nmbug/nmbug | 13 +++++++++++--\r
+ 1 file changed, 11 insertions(+), 2 deletions(-)\r
+\r
+diff --git a/devel/nmbug/nmbug b/devel/nmbug/nmbug\r
+index 81f582c..284d374 100755\r
+--- a/devel/nmbug/nmbug\r
++++ b/devel/nmbug/nmbug\r
+@@ -1,6 +1,6 @@\r
+ #!/usr/bin/env python\r
+ #\r
+-# Copyright (c) 2011-2014 David Bremner <david@tethera.net>\r
++# Copyright (c) 2011-2016 David Bremner <david@tethera.net>\r
+ # W. Trevor King <wking@tremily.us>\r
+ #\r
+ # This program is free software: you can redistribute it and/or modify\r
+@@ -95,7 +95,7 @@ except AttributeError: # Python < 3.2\r
+ _tempfile.TemporaryDirectory = _TemporaryDirectory\r
+ \r
+ \r
+-def _hex_quote(string, safe='+@=:,'):\r
++def _hex_quote(string, safe='+@=:,', encoding='utf-8', errors='strict'):\r
+ """\r
+ quote('abc def') -> 'abc%20def'.\r
+ \r
+@@ -103,6 +103,15 @@ def _hex_quote(string, safe='+@=:,'):\r
+ addition to letters, digits, and '_.-') and lowercase hex digits\r
+ (e.g. '%3a' instead of '%3A').\r
+ """\r
++ if hasattr(string, 'isnumeric'):\r
++ string = string.encode(encoding, errors)\r
++ if hasattr(safe, 'isnumeric'):\r
++ safe_bytes = safe.encode(encoding, errors)\r
++ if len(safe_bytes) != len(safe):\r
++ raise ValueError(\r
++ 'some safe characters are encoded as multiple bytes '\r
++ '({!r} -> {!r})'.format(safe, safe_bytes))\r
++ safe = safe_bytes\r
+ uppercase_escapes = _quote(string, safe)\r
+ return _HEX_ESCAPE_REGEX.sub(\r
+ lambda match: match.group(0).lower(),\r
+-- \r
+2.1.0.60.g85f0837\r
+\r
projects / notmuch-archives.git / commitdiff