Re: gettext's glocale

[Bruno Haible <bruno clisp org>]

Hi Roozbeh,

> 1) What should we do with data conflicts? Let's says an application
> developer wants a simple date displayed in, say, French, and CLDR and
> glibc have different answers for the question?

Both glibc and CLDR have bugzillas, where users can report differences.
But in the meantime, someone has to choose whether to use glibc or CLDR
for the common parts.
  a) the installer/packager/distributor, or
  b) the end user.

a) is implemented through a configure --enable-... option.
b) could be implemented by distributing the binary locale data files for
   both and letting the user set a symlink to one or the other directory
   hierarchy.

Since the installed binary data will be large (around 10 to 15 megabytes)
and b) would basically duplicate its size, my preference is for a).

> Should we have a unified API and only give him glibc data?

That cannot work. The calendar etc. APIs will need CLDR data.

> Or should we have two APIs for giving him whichever he wants? In this
> case, how do we suppose he can find about which API to use for some need
> he has?

If the package author decides whether to use glibc or CLDR data, we get
the worst of all cases: the installer and end user have no freedom of
choice, it's inconsistent even among GNOME applications, and the installed
data on disk has the double size.

> 2) How should we map the CLDR locale system to the glibc locale system
> and vice versa?
>
> While glibc has language, territory, encoding, and a variant, in CLDR
> there are language, script, territory, variant, and some options as key-
> value pairs.
>
> We will need to answer the question anyway, but independently, we can
> answer it our way. If we work together, we should do it much more
> carefully.

The library should handle that internally. We will have to handle
ISO 15924 (http://www.unicode.org/iso15924/iso15924-codes.html) at
some point anyway since it's standard in Windows and MacOS X now.

The end user should have to set a single LC_ALL variable and not
several ones, and its syntax is given by glibc.

Whether the naming of the locale data files on disk should use glibc
conventions or CLDR conventions, I don't know. So far I've been using
glibc conventions for the internal name of a locale, but that can be
reconsidered.

> 3) How do we make it more independent of GNU?
>
> Most of the user-level applications have tried to be portable. GNOME
> runs on Solaris which has its own gettext, Qt and Mozilla don't use LGPL
> libraries where they can be avoided at any cost, etc.

Qt wants to be a platform on their own, including metaobject model,
charset converters, multithreading primitives, QString class, etc. If
they link with libXft, it's - I guess - because they couldn't do the
same library by themselves easily. They don't even link against ICU,
although ICU's license is more like MIT/BSD. Therefore I don't think
they will buy a GNU or GNOME locale library anyway.

KDE is different: They use libintl (or a modified copy of it). I can
well imagine them using an LGPLed libglocale.

Mozilla is different too: The mozilla-1.7.8-gtk2+xft binaries link against
libgtk-x11, libgdk-x11, libgdk_pixbuf, libpangoxft, libpangox, libpango,
libgobject, libgmodule, libglib - lots of LGPLed libraries.

> > It is separate from glibc, because the glibc maintainer didn't
> > acknowledge that having a 'locale_t' data type usable on its own was
> > important. He argues that tying it to the current thread is better.
>
> Do you know his reasons? I'd love to know them.

Ulrich Drepper is very concerned about program startup time, and every
exported symbol in a shared library counts for him. Since uselocale is
already exported from glibc but gettext_l isn't, he prefers programmers
to write

    oldlocale = uselocale(NULL);
    uselocale(mylocale);
    string = dgettext(domain, msgid);
    uselocale(oldlocale);

rather than

    string = dgettext_l(domain, msgid, mylocale);

Whereas in libglocale, I value the ease of use of an API higher than
limiting the number of exported functions.

Similarly, he has put all the locale data files into a single big archive,
to reduce the number of open() calls at program startup.

Bruno