Date: Tue, 12 Jul 2011 21:27:31 +0100
From: Patrick Totzke <patricktotzke@googlemail.com>
To: Carl Worth <cworth@cworth.org>
Subject: Re: Encodings
Message-ID: <20110712202731.GA28929@brick.lan>
References: <87zkkkx6am.fsf@SSpaeth.de> <87box0lv05.fsf@yoom.home.cworth.org>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha1;
	protocol="application/pgp-signature"; boundary="JYK4vJDZwFMowpUq"
Content-Disposition: inline
In-Reply-To: <87box0lv05.fsf@yoom.home.cworth.org>
User-Agent: Mutt/1.5.21 (2010-09-15)
Cc: Notmuch developer list <notmuch@notmuchmail.org>
Precedence: list


--JYK4vJDZwFMowpUq
Content-Type: multipart/mixed; boundary="T4sUOijqQbZv57TR"
Content-Disposition: inline


--T4sUOijqQbZv57TR
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

Hi!

As discussed on irc, if notmuch stores header values in utf8,
its safe to decode them to unicode instances here.
best,
/p


On Mon, Jul 11, 2011 at 08:03:38AM -0700, Carl Worth wrote:
> On Mon, 11 Jul 2011 16:04:17 +0200, Sebastian Spaeth <Sebastian@SSpaeth.d=
e> wrote:
> > The answer is that things are very implicit. notmuch.h speaks of
> > strings but never mentions encodings
>=20
> Much of this was intentional on my part.
>=20
> For example, I intentionally avoided restrictions on what could be
> stored as a tag in the database, (other than the terminating character
> implied by "string" of course).
>=20
> > So, can be document what encoding we are expected to pass in the various
> > APIs
>=20
> Yes, let's clarify documentation wherever we need to.
>=20
> > For some of the stuff we read directly from the files, eg
> > arbitrary headers, we can probably be least sure
>=20
> The headers should be decoded to utf-8, (via
> g_mime_utils_header_decode_text), before being stored in the database.
>=20
> > but are e.g. the returned tags always utf-8?
>=20
> No. The tag data is returned exactly as the user presented it.
>=20
> > I would love to make the python bindings use unicode() instances in
> > cases where we can be sure to actually receive utf-8 encoded strings.
> >=20
> > Encodings make my brain hurt. Unfortunately one cannot simply ignore
> > them.
>=20
> I think a lot of the pain here is due to some bad design decisions in
> python itself. Of course, my saying that doesn't make things any easier
> for you.
>=20
> But do tell me what more we can do to clarify behavior or documentation.
>=20
> -Carl
>=20
> --=20
> carl.d.worth@intel.com



> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch


--T4sUOijqQbZv57TR
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0001-unicode-return-value-for-Message.get_header.patch"
Content-Transfer-Encoding: quoted-printable

=46rom 988a9832d714dfa0f91b2b1185a50acb4a6ca4b5 Mon Sep 17 00:00:00 2001
=46rom: pazz <patricktotzke@gmail.com>
Date: Tue, 12 Jul 2011 19:47:39 +0100
Subject: [PATCH 1/8] unicode return value for Message.get_header()

As discussed in IRC, notmuch recodes mailheaders to
utf-8, so we can safely decode them into unicode instances.
---
 bindings/python/notmuch/message.py |    8 +++++---
 1 files changed, 5 insertions(+), 3 deletions(-)

diff --git a/bindings/python/notmuch/message.py b/bindings/python/notmuch/m=
essage.py
index 763d2c6..4a43a88 100644
--- a/bindings/python/notmuch/message.py
+++ b/bindings/python/notmuch/message.py
@@ -379,14 +379,16 @@ class Message(object):
=20
         :param header: The name of the header to be retrieved.
                        It is not case-sensitive (TODO: confirm).
-        :type header: str
-        :returns: The header value as string
+        :type header: str or unicode instance
+        :returns: The header value as a unicode string
         :exception: :exc:`NotmuchError`
=20
                     * STATUS.NOT_INITIALIZED if the message=20
                       is not initialized.
                     * STATUS.NULL_POINTER, if no header was found
         """
+        if isinstance(header, unicode):
+            header =3D header.encode('utf-8')
         if self._msg is None:
             raise NotmuchError(STATUS.NOT_INITIALIZED)
=20
@@ -394,7 +396,7 @@ class Message(object):
         header =3D Message._get_header (self._msg, header)
         if header =3D=3D None:
             raise NotmuchError(STATUS.NULL_POINTER)
-        return header
+        return header.decode('utf-8')
=20
     def get_filename(self):
         """Returns the file path of the message file
--=20
1.7.4.1


--T4sUOijqQbZv57TR--

--JYK4vJDZwFMowpUq
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iEYEARECAAYFAk4crjMACgkQlDQDZ9fWxarMuQCgjRjhtJGBfVZJIEA4M1f1tS7o
5mEAn1Yp9YhNPIo5N6GZWNiZ53xlyKC2
=XmCG
-----END PGP SIGNATURE-----

--JYK4vJDZwFMowpUq--