Re: Invalid UTF-8 strings

[Chris Vine <chris cvine freeserve co uk>]

On Fri, 2007-06-01 at 13:07 -0400, Matt Bragano wrote:
> Great debugging techniques, thanks!
> 
> Turns out one of the strings being rendered contained some non-ASCII 
> characters. The functions:
> 
> Glib::locale_to_utf8()
> Glib::locale_from_utf8()
> 
> did the trick.  In order to render the text in a tree view I had to 
> convert to UTF-8, but to open a directory using Glib::Dir() I had to 
> convert back to the locale. 
> 
> Makes perfect sense now :-)

Yes, you need to pass only UTF-8 strings to most GTK+/pango functions
(see below for the exceptions), but be careful: you always need to know
what the codeset is that text you are handling employs when doing a
codeset conversion, including when using Glib::locale_to_utf8().  If the
locale on your machine is not a UTF-8 locale, and you apply
Glib::locale_to_utf8 to text already in UTF-8 encoding, you will get
another conversion error.

To convert filenames you should really use Glib::filename_to_utf8() and
Glib::filename_from_utf8(), because the file system may use a different
encoding for filenames than the locale codeset, particularly if network
file systems are being used.  (See also the G_FILENAME_ENCODING
environmental variable, which Glib::filename_to_utf8() and
Glib::filename_from_utf8() will use, and the G_BROKEN_FILENAMES
environmental variable, which if set will cause Glib to assume that the
filesystem locale is the same as the local machine locale.
G_FILENAME_ENCODING has precedence over G_BROKEN_FILENAMES - the
G_BROKEN_FILENAMES name comes from the possibly high-handed belief of
the Glib authors that using a machine-local codeset for a filesystem in
the network aware age is "broken").  Note also that a number of GTK+
file handling functions take or return the filesystem codeset and not
UTF-8 (which was presumably the problem that you had).

Chris