MIME-Version: 1.0
Sender: amdragon@gmail.com
In-Reply-To: <1315972539.2201.11.camel@delen>
References: <1315972539.2201.11.camel@delen>
Date: Thu, 15 Sep 2011 13:52:12 -0400
Message-ID:
 <CAH-f9WtL4Lwrf2qSzpgeLL5nA_2_mFxUm6cFLmfO9UK_aKmCkg@mail.gmail.com>
Subject: Re: Unicode Paths
From: Austin Clements <amdragon@mit.edu>
To: Martin Owens <doctormo@gmail.com>
Content-Type: text/plain; charset=ISO-8859-1
Cc: Notmuch developer list <notmuch@notmuchmail.org>
Precedence: list

On Tue, Sep 13, 2011 at 11:55 PM, Martin Owens <doctormo@gmail.com> wrote:
> Hello Again,
>
> I notice in the lib code notmuch_database_open(),
> notmuch_database_create() these functions use const char *path for the
> directory path input. Is this unicode safe?
>
> The python bindings (and ctype docs) seem to suggest using something
> called 'wchar_t *' for accepting unicode but that's for C not C++.
>
> Is this something that should be patched?

char* is the correct type for paths on POSIX systems.  The *meaning*
of those bytes is a more complicated matter and depends on your locale
settings.  On old systems it was generally ASCII, on modern systems
it's generally UTF-8, and it can be many other things.  However, as a
consequence of UNIX's C heritage, it is *always* terminated with a
NULL byte and cannot contain embedded NULL's.  Any encoding that
doesn't satisfy this would not be a valid encoding for file names (you
couldn't even pass such a file name to the open() system call, because
it expects a NULL-terminated byte string).

wchar_t is another matter entirely.  wchar_t is the type used by C to
represent wide strings internally, which generally (but not
necessarily!) means it stores a Unicode code point.  However, this
isn't an encoding, and different compilers can give wchar_t different
meanings, so wchar_t strings aren't generally appropriate for storing
or sharing between processes or with the kernel.