Re: text representation in gtk+ 1.3.x

[Owen Taylor <otaylor redhat com>]

Pablo Saratxaga <pablo mandrakesoft com> writes:

> Kaixo!
> 
> On Tue, Dec 12, 2000 at 11:14:48PM -0500, Havoc Pennington wrote:
> 
> > Hi,
> > 
> > I think the plan is to just run a perl script that converts all the
> > .po files to be in UTF-8.
> 
> I don't think doing that automatically is a good idea, a lot of po files
> can be completly distroyed doing that: those using 8bit encodings not
> yet recognized by iconv(), or whose tables have small bugs, or those po
> files which have non-ascii in the msgid (in people's names for example)
> and use for the msgstr a different character set than the one used
> for the msgid.
> using iconv() on them will truncate the file at the first unknown char.

Well, maybe it needs a little manual checking, or maybe the 
script needs a little enhancement, but doing it in a basically
automated fashion is the right thing to do.

I'm not really worried to much about 8-bit encodings not recognized
by iconv - there simply aren't all that many po files for those
languages -they can be handled as a special case.
 
> For Chinese and Japanese (and any language not using spaces) there is also
> the risk of producing unreadable stuff by breaking lines at the middle of
> a char (as the conversion to utf-8 will increase the lenght of the strings
> in the msgstr, the next msgmerge can cut them at the wrong place 

It should be easy to make the Perl script handle rebreaking such
strings - there is no need to make the segmentation linguistically
correct, just to make sure that it doesn't split UTF-8 characters.

> (that is a general problem of gettext tools in fact; but also of the
> Gtk+ word wrapping algorithm (unless it has been solved recently, on
> the versions I use a Chinese or Japanese line is not wrapped if no
> spaces are present, that may lead to dialog boxes several times
> wider than the screen...)

We have code now in Pango that should handle things well for all
but a few languages. (Basically languages that require dictionary-based
line breaking.)

For GTK+-1.2, there is a patch from Akira Higuchi that improves things
quite a bit at least for Japanese - we ship this as part of the Red
Hat RPM, but I've never applied it to the main distribution of GTK+,
because I've had some doubts as to how it does on systems with
less-functional locale support.

Regards,
                                        Owen