Re: UTF8 support

[Pavel Roskin <proski gnu org>]

Hello!

> New version of RedHat has unicode console and problems with Midnight
> Commander in cyrillic mode. I try to use new version 4.6pre1, but
> problem exist too. Last version of ncurses has support of UTF-8, but it
> requires some changes of source code.

What changes are needed?  Is it possible to stay compatible with
non-unicode terminals if using ncurses with UTF-8 support?

> Whether you plan full support of UTF-8 console mode in Midnight
> Commander? I think that it don't demands big changes...

I don't know.  Many people are asking this question, but nobody is working
on it.  I haven't seen any working patch for UTF-8 support, so I cannot
say if the changes are going to be big.

I believe that the changes will be significant for several reasons.

1) Right know, mc is not aware of the output encoding unless charset
conversion code is used.  The text is shown on the screen without
conversion, as is.  UTF-8 support would require that the charset
conversion code is enabled.  There are some issues with that code, and
they should be resolved before it is enabled by default.

2) Sizes of all visual elements are calculated in the assumption that one
byte corresponds to one character.  That's true for ASCII and it may be
true for some Asian languages with wide characters, but it's not true for
European non-ASCII characters (including upper part of ISO-8859-1,
ISO-8859-2, Greek and Cyrillic).  Fixing this would require changing
almost every strlen() to something Unicode-aware.

3) The hot keys are currently indicated by "&" in the text.  The next
character is the hot key.  Now the hot key will have to be represented in
unicode.  To interpret it, mc will have to know the input encoding, which
is not always the same as the output encoding.

4) The same binary compiled with UTF-8 support must work on non-UTF
terminals, e.g if somebody logs in remotely.

5) The same source should compile on the systems without iconv support,
which means adding fallbacks to strlen().

If you know a simple way how to do all that, please explain.

-- 
Regards,
Pavel Roskin