Re: normalizing filenames and strings

[Xavier Bestel <xavier bestel free fr>]

On Thu, 2007-03-29 at 09:52 +0200, Alexander Larsson wrote:
> On Wed, 2007-03-28 at 19:52 -0600, Federico Mena Quintero wrote:
> 
> > If you have applications that "don't work" when you have Unicode
> > filenames, it's a telltale sign that they are not using
> > g_filename_to_utf8() and g_filename_from_utf8() correctly:
> > 
> > http://developer.gnome.org/doc/API/2.0/glib/glib-Character-Set-Conversion.html#file-name-encodings
> > 
> > I'd be interested in knowing of applications that use this API properly,
> > but that still have problems with composed/decomposed names.  In that
> > case, we may need to explicitly normalize inside those functions:  they
> > are the central point of change between UTF-8 (GNOME's encoding for
> > strings) and the file system's own encoding.

I think the expectation is false. GNOME apps shouldn't treat these
filenames as equivalent, exactly like "Makefile" and "makefile" should
be 2 different filenames, except when completing.

> > [Those functions don't normalize currently; maybe they need to... do you
> > have a reproducible bug?]
> 
> NO! They should not normalize! Then you can't open a file that has an
> unnormalized filename.

And that may introduce security bugs: if you transform a filename behind
the user's back, you have to be pretty sure of your utf8 decoder - see
http://www.cl.cam.ac.uk/~mgk25/ucs/examples/UTF-8-test.txt for examples
of subtilities.
Windows IIS even had a security bug where the attacker crafted an URL
with special characters with alternative (illegal) unicode encoding,
which wrongly passed safety tests but once canonicalized enabled access
to system paths (and even remote command execution, cf
http://seclists.org/bugtraq/2000/Oct/0271.html for details).

Xav