Re: Font lookup ranges [was Re: Notes on Pango Xft backend]

[Keith Packard <keithp keithp com>]

Around 18 o'clock on Jun 3, Owen Taylor wrote:

> Hmmm, limiting font tag usage to CJK definitely would reduce the
> possible harm that could result; and probably makes sense. (At
> least initially.)

I'm hoping that CJKV will remain the only "broken" part of Unicode in the 
future; it appears that the Unicode group has become a bit more sensitive 
to local objections over unification.  The expansion from 16 to 24 bits 
has given them a lot more flexibility.

If we could come up with a mechanism that left this bit of policy down in 
fontconfig, that would centralize any future extensions.  That isn't what 
we're doing here though.

> This is fine; I thought you were suggesting something that
> would involve conditionalizing on language in fonts.conf.

Hmm.  We could do something like:

	if (language_tag not member (japanese, traditional, 
				     simplified, korean, vietnamese))
		delete language_tag

in the fonts.conf file -- this would place the policy over how language 
tags affect font selection down inside fontconfig which would reduce the 
number of places fixes would need to be applied if the set of necessary 
language tags ever increases.

>  - They pull up a Traditional Chinese document (not itself tagged) 
>    in their word processor and see that it is a mix of MS Gothic and 
>    AR PL KaitiM Big5.
>  
>  - They select "AR PL KaitiM Big5" from the font menu.
> 
>  - The display doesn't change because Xft is still finding 
>    MS Gothic which matches the language tag.

It matches the locale language tag, but the document doesn't have a 
language tag.  Are you suggesting that the locale language tag be used for 
documents which have no language tag?  Hmm.  This is getting complicated.

Let's look at what happens down inside fontconfig and see if we can't make 
some sense.

>  - They pull up a Traditional Chinese document (not itself tagged) 
>    in their word processor and see that it is a mix of MS Gothic and 
>    AR PL KaitiM Big5.

With an unspecified family, fontconfig will build the pattern:

	family:		"sans-serif"

> 	<alias>
> 		<family>sans-serif</family>
> 		<prefer>
> 			<family>MS Gothic</family>
> 			<family>AR PL KaitiM Big5</family>
> 			<family>Norasi</family>
> 			<family>Garuda</family>
> 			<family>Arial Unicode MS</family>
> 		</prefer>
> 	</alias>

This alias will edit the pattern to:

	family:		"MS Gothic", "AR PL KaitiM Big5", "Norasi", "Garuda",
			"Arial Unicode MS", "sans-serif"

In the absense of language tags, this will prefer MS Gothic and fall back
to AR PL KaitiM Big5 for glyphs missing from that.  This depends on the 
correct ordering of fonts within this alias; something which seems a bit 
fragile.

>  - They select "AR PL KaitiM Big5" from the font menu.

Now the pattern changes:

	family:		"AR PL KaitiM Big5", "sans-serif"

The edit converts this to:

	family:		"AR PL KaitiM Big5", "MS Gothic", "AR PL KaitiM Big5",
			"Norasi", "Garuda", "Arial Unicode MS", "sans-serif"

In the absense of language tags, this will match the Big5 font.

If we apply the locale language tag when it represents a Han language, 
then we'll *always* pick MS Gothic; the language tag has higher precedence 
than the font family in the selection process.  This is clearly wrong; the 
users font specification should always be more important than a 
language-tag driven fallback.

Somehow, we need to use the language tag when selection which fallback 
font to pick, but not when choosing between "real" family names provided 
by the application.

Hmm.  It feels like there is a cut-over in the list of families; the first 
part of the list is "language tag independent" -- family names provided by 
the application should normally be preferred to families selected by 
language tag, family names used as fallbacks should be ordered by language 
tag fit.

Here's a suggestion (please feel free to knock holes in it):

Tag entries in the family list as to whether they're language-tag selected 
or not language-tag selected (or perhaps just whether they're "fallback" 
or "non-fallback" entries).  Edits relative to those entries are 
contaminated and the resulting entries inherit that state.  Now the config 
file tags the 'sans-serif' alias as a 'fallback' entry; now those entries 
are matched based on the distance from the language tag (as well as the 
order within the list).

I am still planning on adopting pure CSS2 matching for fontconfig -- I 
have a sense that CSS2 matching would solve this problem; I'll go and read 
that spec again and try to make sense of this.  If it does, it may be that 
we should always supply the language tag to fontconfig and expect that the 
matching rules will generate a reasonable match.

Keith Packard        XFree86 Core Team        HP Cambridge Research Lab