Comprehensive East-Asian support

[Steve Underwood <steveu coppice org>]

Hi all,

Last summer I exchanged several private e-mails with Owen Taylor about
gscript. There was nothing private about the discussion - I just failed
to join this list, despite sending several subscribe requests. At that
time Owen's goals for gscript were somewhat narrow - basically just to
display text within the GTK display environment. I said that I thought
it important for gscript to provide support for vertical and
right-to-left text. Owen didn't see that as important for dialogs, and
similar GTK related things, which may well have been the correct
attitude in the context of July 1999's gscript. January 2000's Pango is
intended as a system for the comprehensive handling of international
high quality text output, so its time for me to restate my point of
view.

A system which can only handle the text direction features defined in
the Unicode support tables, bi-directional algorithm, etc. is too
limited for East Asian languages. Good Chinese and Japanese (I can't
speak for Korean, as I don't see that much of it) support demands
support for top-to-bottom, right-to-left text. Chinese also demands
options for left-to-right and right-to-left (I'm not sure if Japanese is
ever written right-to-left, now or in the past). The Unicode tables
assume all East Asian languages are only ever written left-to-right.

As well as the need to properly present blocks of text, the behaviour of
international systems when labelling the y-axis of a graph, and similar
rotated text situations is important. Rendering according to the Unicode
rules, and rotating the outcome gives the wrong result for East Asian
languages. Kanji/Hanzi are _never_ tilted beyond about 45 degrees when
labelling anything. Turning them on their side, along with some numbers
or English words they may be mixed with would be totally wrong.

Even for left-to-right Chinese and Japanese, the Unicode material, and
the current gscript/pango, fail to implement things Asian people would
consider important in high quality output. For example, if a short burst
of Hanzi is dropped into a page of English it would be rendered just as
gscript does now. If a short burst of English (say a company name) were
dropped into a page of Chinese, space would be placed around the English
so the Chinese characters all sit on a mon-spaced grid. Pango doesn't
seem to provide for that, and it leads to very odd looking results.

If facilities for rotated text for labelling purposes, and alternative
directions for text blocks are not put into the Pango API I think they
are going to get hacked in later by some folk in Asian - possibly with
me amongst them. It seems better to include the proper provision for
comprehensive text handling from day one. The code that needs to lie
behind the API could be missing from version 1 of Pango, but I strongly
feel the API needs to make provision for these things, or an
incompatible update will occur later.

If folk are interested in providing this capability, I will be happy to
document (which basically means gather illustrative examples) the
detailed behaviour East Asian people have come to expect of mixed
language handling in vertical and rotated text. I might also have some
time to help in the implementation.

Last summer Owen seemed convinced that people don't use vertical or
right-to-left text any more. That puzzled me, since I know he can read
Japanese. If people aren't convinced, I can post a few small scanned
images of Hong Kong and Japanese newspapers and magazines to prove that
vertical is still the order of the day in Asia. I can pick almost any
newspaper and any page, and show vertically written text.

Steve