Re: GError location filename as wide chars

[Quentin Sculo <squentin free fr>]

On Thu, 2009-02-12 at 10:18 +1100, Kevin Ryde wrote:
> Torsten Schoenfeld <kaffeetisch gmx de> writes:
> > Did you take a look at the filename conversion stuff in Glib
> > (filename_to_unicode, filename_from_unicode) too, by chance?  Marc
> > Lehmann says it's totally broken:
> > <https://rt.cpan.org/Public/Bug/Display.html?id=39438>.
>
> I might have seen that a while ago, and couldn't understand.
>
> Having a wild guess: if gperl_filename_from_sv() gets an sv with the
> utf8 flag off, then return the contents directly as the bytes, instead
> of now spinning it up with sv_utf8_upgrade and down again with
> g_filename_from_utf8.
>
> The current sv_utf8_upgrade I guess interprets the bytes as latin1
> always, so if your filename locale is not latin1 it won't be the same
> when it comes back down to bytes.  Does that sound likely?  Is it the
> claimed problem?  Does anyone want latin1 strings as filenames turned
> into utf8 if their locale is utf8?  :-)

The problem is much worse than that, filenames in linux/unix are just
byte streams, there is no enforced encoding. So the "filename locale" is
just an indication, but the filenames can still be anything, and so
should be left as is. You can't expect to convert a filename to utf8 and
still have something that can be used to open the file.

The way filenames are handled in Glib/Gtk2 leads to a few problems when
trying to use filenames that have a wrong encoding, in particular in
FileChooserDialog (and the deprecated FileSelection). It's also
impossible to use Gtk2::Gdk::Pixbuf->new_from_file with these files, I
work around this by reading the file myself and feeding it to a
PixbufLoader.

But I fear it's not possible to fix this without breaking existing
code :(

Quentin