Date: Tue, 16 Feb 2016 09:56:42 -0800
From: "W. Trevor King" <wking@tremily.us>
To: David Bremner <david@tethera.net>
Cc: notmuch@notmuchmail.org
Subject: Re: [PATCH] nmbug: Allow Unicode tags and IDs in Python 2
Message-ID: <20160216175641.GN4265@odin.tremily.us>
References:
 <e287050a10ce1d2120db996d2d200f610370a44e.1455513965.git.wking@tremily.us>
 <87lh6kvmbc.fsf@zancas.localnet>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha1;
 protocol="application/pgp-signature"; boundary="b1CVx77D595wdcW8"
Content-Disposition: inline
In-Reply-To: <87lh6kvmbc.fsf@zancas.localnet>
OpenPGP: id=39A2F3FA2AB17E5D8764F388FC29BDCDF15F5BE8;
 url=http://tremily.us/pubkey.txt
User-Agent: Mutt/1.5.23 (2014-03-12)
Precedence: list


--b1CVx77D595wdcW8
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Tue, Feb 16, 2016 at 09:04:07AM -0400, David Bremner wrote:
> W. Trevor King writes:
> > Coercing to UTF-8 (regardless of locale) gives us consistent tag
> > IDs for sharing between users.
>=20
> I'm not sure what "tag IDs" are. Do you mean message-ids here? or "tags
> and IDs"?

Yeah.  I'll fix that in v2.

> At first I thought there might be problems with non-utf8 message-ids,
> but that turns out not to be the case [1].  It seems like it would take
> a fairly heroic effort to get non-UTF8 tags into the database (perhaps
> by calling the library interface with bad strings?) so we can probably
> ignore this case. It might be good to document the limitation though,
> since AFAIK, dump and restore can roundtrip any old crap.

How about in a NEWS entry in v2 of this series, and then echoing that
NEWS entry in the notmuch-dtags (or whatever) man page once I work up
that series?

> > The 'isnumeric' check identifies Unicode instances in both Python
> > 2 [9] and Python 3 [10].
>=20
> I still haven't really tried to understand this part, but probably
> it deserves inline documentation.

It's just =E2=80=9Cif you have a Unicode instance (str in Python 3, unicode=
 in
Python 2), convert it to bytes (bytes in Python 3, str in Python 2).
Only Unicode instances will have an =E2=80=98isnumeric=E2=80=99 method, so =
it's a
convenient marker for switching that logic.  I'll add a =E2=80=9Cconvert fr=
om
Unicode if necessary=E2=80=9D comment to v2.

> > I haven't checked the other commands for issues with Unicode IDs
> > or tags.  It's possible that in addition to this explicit encoding
> > to UTF-8, we'll also want explicit decoding from UTF-8 when
> > reading from Git trees (for 'nmbug checkout' and 'nmbug status').
>=20
> Yes, this seems to be a problem, with the patch applied I can
> commit, but the same utf-8 message-id causes problems.

Ugh.  Thanks for checking.  I'll try to fix all the places where this
would have an impact in v2 of this series.

Cheers,
Trevor

--=20
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

--b1CVx77D595wdcW8
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBAgAGBQJWw2LXAAoJEAPqygegUbGsRC8P/2Ao8dkv4CzkN95Fr6k5t/yR
3Y231DolowEHfS11uHE7JwD0XuQrM7K/Z8orm692Q8QaoNb6iAuf3KvtIoJKUWZB
wJc4Fhv8MDNzeyNmCtLMGZw3u/Cm5OTL513c154qUtcrkO6WoQWknKQqzNLViamr
5zWzD/w0tW8dZHJWWntOHyz73mbC0E7Ib4UoEUJ06BWJKCel1qv8TumtsDr0sh+e
F7mNMyZwnE85LHOePBPkwNedtjOq4fQ9xfma5moN3rZU3owYXkJUXcG8TqCzm8X4
kJaSUZv8B9ig9uyS7yBN9Gp7B4EMuKrI8k2PM/sDv9p/IncHsGZPCqf/skeoBHUg
bnoget1HjpOiTjEau4gB+DPGrxEOCHbzt50USM12/6vIRVhQmWKZcV8kLA5jtA+V
91dEagZjMWavxXUr07E3YP4dzo3PH/vsPCLA5aaJVpbiIEIq/xm/J7QHkVTro66E
sCpag5SjFDk3lkN4cvmDBWRF/VzT58qbQ+NM1nMg4Ydfiu+mmXZSe907EnBweRQd
ffzoQpN7rubP4QLpVrVAr/kHB6sXYNJOMSn7SS4Dul5bLQOwk7LeqZZEQjA6B8Cb
MeSXwIfJuoY+rnSdeaTFjvtF5c6Ri85ptQMpPnShA3u66ym9k8gDOINQOXsr/Max
Sdg9K0czco6hXUfIcZN5
=bRRC
-----END PGP SIGNATURE-----

--b1CVx77D595wdcW8--