Language tags



The last major API bug I'm planning to deal with for Pango before 1.0
is to clean up the language tag handling.

The idea of a language tag is that it identifies the language
of a portion of text.

The primary use of language tags in Pango is for taking lanugage ito
account when shaping - for instance, to choose between simplified and
traditional chinese character shapes. 

A second use of language tags is when determining metrics for a font
 -- the metrics for handling all possible languages with all characters
in a Unicode font or fontset can be very big -- Arabic characters, in
particular, tend to be substantially wider and have bigger descenders
than characters in other languages. So, by providing a language tag to
pango_font_get_metrics() we provide a hint about what characters
we are interested in and expect to get.

There is a default language tag for each PangoContext, and language
tags can also be set as pango attributes for each run. Language tags
also appear in the Pango API in a few other places - in particular
pango_font_get_metrics().

I indentified three distinct problems with the way that Pango
currently handles language tags:

 1) There is no exact specification of what a language tag is,
    or how to match language tags against patterns. There was
    a rough idea that language tags would look like 'en_US',
    and that would somehow match 'en' as well.

 2) Language tags are not available at shaping time. While language
    tags are taken account when choosing a shaper, if a shaper
    handles multiple languages (like the basic shaper) it doesn't
    have access to information about language tags for individual
    runs of text.

 3) Since language tags are treated as strings, there is a lot
    of copying of language tags, malloc, and strcmp() involved.


Looking around a bit, the solution to 1) was pretty easily to 
hand - a specification for language tags can be found in RFC 3066;
these language tags, which are of the form:

 en-us
 zh-hakka
 zh-min-nan
 
Are used for http, html, xml, mail, and are the form of language tag
currently supported by the Unicode plane 14 language tag
characters. So, using these tags will give wide application
compatibility. They also are compatible with the rough idea of 'en_US'
being a language tag - if the first component of a language tag is 2
characters, it must be a ISO 639-1 language code; if the second
component of the tag is 2 characters, it must be a ISO 3166 country
code.

RFC 3066 includes a specification for language ranges - a language
range is either '*' which matches all language tags, or a prefix such
as 'en' which matches all language tags starting with those
components.

The main difficulty with RFC 3066 language tags is that they are
specified to be case insensitive, so matching is complicated a
bit. I'd also like to support, as an extension, matching 'en_US' to
'en-us' to provide a bit more compatibilty with Unix locales.


The second problem was quite easy to address - the approach I'm
taking for this one, is to simply add a language tag to the
PangoAnalysis structure. Since the PangoAnalysis structure is 
passed to the shaper, the shaper then has the necessary information.
Doing this does make the third problem - efficiency - more
important. Since we are storing a language tag for each run,
we need to be able to store a language tag quickly - hopefully
without needing to malloc an additional string.


The basic idea for handling the question of efficiency for copying
and comparing language tags was one of canonicalization - taking
a string representing a language tag and converting it to canonical
form - a characte that can be matched as a string without need
for case comparisons, and can be compared as a pointer for
equality with other canonicalized language tags.

  /**
   * pango_lang_canonicalize:
   * @lang: a string representing a language tag
   * 
   * Take a RFC-3066 format language tag and canonicalize it
   * by converting it to lowercase, mapping '_' to '-', and stripping
   * all characters other than letters and '-'. Then the result
   * is looked up in a central database, and a canonical value
   * for the result is returned, that can be compared for equality
   * with the results of other calls to pango_lang_canonicalize()
   * 
   * Return value: the canonical form of the language tag. 
   **/
  G_CONST_RETURN char *pango_lang_canonicalize (const char *lang);

Copying a canonical language tag doesn't require anything but copying
the pointer. The reason for going with this approach rather than using
integer id's (something like GQuark) is that we do need to do
efficient string operations on language tags - in particular for
matching a language tag on a range.

The main problem found here is that it isn't clear from the
API when a canonicalized language tag is needed, and when a 
normal tag is needed. Currently, in the API:

 - The language tags for pango_context_set_lang(),
   pango_attr_lang_new(), and pango_font_get_metrics() don't have
   to be canonicalized.

 - The language tag in the PangoAnalysis structure and 
   for "internal" functions like pango_font_get_converage() must
   be canonicalized.

But in all these cases, the language tag is represented as a
char *.

An obvious approach would be to introduce a typedef, either
transparent:

 typedef const char *PangoLangTag;

Or (probably better) opaque:

 typedef struct _PangoLangTag *PangoLangTag;

 #define pango_lang_tag_to_string(tag) ((const char *)(tag))

The main problems with adding such a typedef are:

 - It makes things a little more complex for users of the 
   public API's since language tags are no longer just strings - the 
   users has to call pang_lang_tag_from_string() [ a rename
   of pango_lang_canonicalize ]
   
 - It doesn't map into other languages very well - it is somewhat
   outside the realm of what is standard in our API's. 

   But OTOH, the opaque typedef can be thought of as a standard
   "boxed type", and the canonicalized strings in the "private"
   APIs _definitely_ don't map into other languages at all - 
   canonicalization will be lost when converting to the other
   language and back.

Attached is the current version of the patch - which does
canonicalization without introducing a typedef. It also
introduces:

  /**
   * pango_lang_matches:
   * @lang: a RFC 3066 language tag, canonicalized as by pango_lang_canonicalize()
   * @range_list: a list of language ranges, separated by ';' characters.
   *   each element must either be '*', or a RFC 3066 language range
   *   canonicalized as by pango_lang_canonicalize().
   * 
   * Checks if a language tag matches one of the elements in a list of
   * language ranges. A language tag is considered to match a range
   * in the list if the range is '*', the range is exactly the tag,
   * or the range is a prefix of the tag, and the character after the
   * tag is '-'.
   **/
  gboolean
  pango_lang_matches (const char *lang,
                      const char *range_list);

Regards,
                                        Owen

Index: ChangeLog
===================================================================
RCS file: /cvs/gnome/pango/ChangeLog,v
retrieving revision 1.273
diff -u -r1.273 ChangeLog
--- ChangeLog	2001/06/08 16:02:58	1.273
+++ ChangeLog	2001/06/09 22:36:30
@@ -1,3 +1,19 @@
+Sat Jun  9 17:36:09 2001  Owen Taylor  <otaylor redhat com>
+
+	* pango/itemize.c: Remove old, unused file.
+
+Sat Jun  9 15:05:34 2001  Owen Taylor  <otaylor redhat com>
+
+	* pango/pango-context.c (pango_itemize): Reduce number of mallocs
+	by allocating one array of PangoAnalysis instead of many arrays.
+
+	* pango/pango-attributes.[ch] (pango_attr_iterator_get_font): Return
+	the language tag as well.
+
+	* pango/pango-item.[ch] pango/pango-layout.c: Move extra_attrs
+	from PangoItem to PangoAnalysis. Add a language tag field to
+	PangoAnalysis. (#55894)
+
 Thu Jun  7 14:06:25 2001  Owen Taylor  <otaylor redhat com>
 
 	* pango/pango-layout.c (pango_layout_move_cursor_visually): 
Index: modules/basic/basic-x.c
===================================================================
RCS file: /cvs/gnome/pango/modules/basic/basic-x.c,v
retrieving revision 1.23
diff -u -r1.23 basic-x.c
--- modules/basic/basic-x.c	2001/04/27 23:50:00	1.23
+++ modules/basic/basic-x.c	2001/06/09 22:36:30
@@ -28,7 +28,9 @@
 
 typedef struct _CharRange CharRange;
 typedef struct _Charset Charset;
+typedef struct _CharsetOrdering CharsetOrdering;
 typedef struct _CharCache CharCache;
+typedef struct _CharCachePointer CharCachePointer;
 typedef struct _MaskTable MaskTable;
 
 typedef PangoGlyph (*ConvFunc) (CharCache   *cache,
@@ -47,6 +49,12 @@
   ConvFunc conv_func;
 };
 
+struct _CharsetOrdering
+{
+  const char *langs;
+  char charsets[MAX_CHARSETS];
+};
+
 struct _CharRange
 {
   guint16 start;
@@ -64,10 +72,18 @@
 
 struct _CharCache 
 {
+  guint ref_count;
+  CharsetOrdering *ordering;
   MaskTable *mask_tables[256];
   GIConv converters[MAX_CHARSETS];
 };
 
+struct _CharCachePointer
+{
+  const char *lang;
+  CharCache *cache;
+};
+
 static PangoGlyph conv_8bit (CharCache  *cache,
 			     Charset    *charset,
 			     const char *input);
@@ -112,17 +128,37 @@
  * X window system script engine portion
  */
 
+/* Structure of our cache:
+ *
+ * PangoFont => CharCachePointer  ===\
+ *                    |               \
+ *              CharCachePointer  ======> CharCache => CharsetOrdering
+ *                    |                       |======> MaskTable[0]    => {subfonts,charset}[n_subfonts], 
+ *                    |                       |======> MaskTable[1]    => {subfonts,charset}[n_subfonts], 
+ *                    |                       \======> MaskTable[...]  => {subfonts,charset}[n_subfonts]
+ *                    |
+ *              CharCachePointer  ======> CharCache => CharsetOrdering
+ *                                            |======> MaskTable[0]    => {subfonts,charset}[n_subfonts], 
+ *                                            |======> MaskTable[1]    => {subfonts,charset}[n_subfonts], 
+ *                                            \======> MaskTable[...]  => {subfonts,charset}[n_subfonts]
+ * 
+ * A CharCache structure caches the lookup of what subfonts can be used for what characters for a pair of a Font
+ * and CharsetOrdering. Multiple language tags can share the same CharsetOrdering - the list of CharCachePointer
+ * structures that is attached to the font as object data provides lookups from language tag to charcache.
+ */
 static CharCache *
-char_cache_new (void)
+char_cache_new (CharsetOrdering *ordering)
 {
   CharCache *result;
   int i;
 
   result = g_new0 (CharCache, 1);
 
+  result->ref_count = 1;
+  result->ordering = ordering;
   for (i=0; i<MAX_CHARSETS; i++)
     result->converters[i] = (GIConv)-1;
-
+  
   return result;
 }
 
@@ -188,10 +224,12 @@
 
       for (i=0; i<G_N_ELEMENTS(charsets); i++)
 	{
-	  if (mask & (1 << i))
+	  int charset_index = cache->ordering->charsets[i];
+	  
+	  if (mask & (1 << charset_index))
 	    {
-	      charset_names[n_charsets] = charsets[i].x_charset;
-	      charsets_map[n_charsets] = &charsets[i];
+	      charset_names[n_charsets] = charsets[charset_index].x_charset;
+	      charsets_map[n_charsets] = &charsets[charset_index];
 	      
 	      n_charsets++;
 	    }
@@ -329,19 +367,90 @@
     }
 }
 
+static void
+char_caches_free (GSList *caches)
+{
+  GSList *tmp_list = caches;
+  while (tmp_list)
+    {
+      CharCachePointer *pointer = tmp_list->data;
+      
+      pointer->cache->ref_count--;
+      if (pointer->cache->ref_count == 0)
+	char_cache_free (pointer->cache);
+      g_free (pointer);
+      
+      tmp_list = tmp_list->next;
+    }
+  g_slist_free (caches);
+}
+
+static CharsetOrdering *
+ordering_for_lang (const char *lang)
+{
+  int i;
+
+  for (i = 0; i < G_N_ELEMENTS (charset_orderings) - 1; i++)
+    {
+      if (pango_lang_matches (lang, charset_orderings[i].langs))
+	return &charset_orderings[i];
+    }
+  
+  return &charset_orderings[i];
+}
+
 static CharCache *
-get_char_cache (PangoFont *font)
+get_char_cache (PangoFont  *font,
+		const char *lang)
 {
   GQuark cache_id = g_quark_from_string ("basic-char-cache");
-  
-  CharCache *cache = g_object_get_qdata (G_OBJECT (font), cache_id);
-  if (!cache)
+  CharCache *cache = NULL;
+  CharCachePointer *pointer;
+  CharsetOrdering *ordering;
+  GSList *caches;
+  GSList *tmp_list;
+
+  caches = g_object_get_qdata (G_OBJECT (font), cache_id);
+  tmp_list = caches;
+  while (tmp_list)
     {
-      cache = char_cache_new ();
-      g_object_set_qdata_full (G_OBJECT (font), cache_id, 
-			       cache, (GDestroyNotify)char_cache_free);
+      pointer = tmp_list->data;
+      if (pointer->lang == lang)
+	return pointer->cache;
+
+      tmp_list = tmp_list->next;
     }
 
+  ordering = ordering_for_lang (lang);
+
+  tmp_list = caches;
+  while (tmp_list)
+    {
+      pointer = tmp_list->data;
+      if (pointer->cache->ordering == ordering)
+	{
+	  cache = pointer->cache;
+	  break;
+	}
+
+      tmp_list = tmp_list->next;
+    }
+
+  if (!cache)
+    cache = char_cache_new (ordering);
+  else
+    cache->ref_count++;
+
+  pointer = g_new (CharCachePointer, 1);
+  pointer->lang = lang;
+  pointer->cache = cache;
+
+  caches = g_slist_prepend (caches, pointer);
+  
+  g_object_steal_qdata (G_OBJECT (font), cache_id);
+  g_object_set_qdata_full (G_OBJECT (font), cache_id, 
+			   caches, (GDestroyNotify)char_caches_free);
+
   return cache;
 }
 
@@ -363,7 +472,7 @@
   g_return_if_fail (length >= 0);
   g_return_if_fail (analysis != NULL);
 
-  cache = get_char_cache (font);
+  cache = get_char_cache (font, analysis->lang);
 
   n_chars = g_utf8_strlen (text, length);
   pango_glyph_string_set_size (glyphs, n_chars);
@@ -454,7 +563,7 @@
 basic_engine_get_coverage (PangoFont  *font,
 			   const char *lang)
 {
-  CharCache *cache = get_char_cache (font);
+  CharCache *cache = get_char_cache (font, lang);
   PangoCoverage *result = pango_coverage_new ();
   gunichar wc;
 
Index: modules/basic/tables-big.i
===================================================================
RCS file: /cvs/gnome/pango/modules/basic/tables-big.i,v
retrieving revision 1.6
diff -u -r1.6 tables-big.i
--- modules/basic/tables-big.i	2000/12/15 01:10:10	1.6
+++ modules/basic/tables-big.i	2001/06/09 22:36:30
@@ -42,6 +42,13 @@
   { 18, "ISO-10646",    "iso10646-1",      conv_ucs4 }
 };
 
+CharsetOrdering charset_orderings[] = {
+	{ "zh-cn", { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 18, 15, 17 } },
+	{ "zh-tw", { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 18, 15, 17 } },
+	{ "ja", { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 18, 16, 17 } },
+	{ NULL, { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 18, 15, 16, 17 } },
+};
+
 const guint32 char_mask_map[] = {
   0,
   ENC_ISO_8859_10|ENC_ISO_8859_1|ENC_ISO_8859_13|ENC_ISO_8859_14|ENC_ISO_8859_15|ENC_ISO_8859_2|ENC_ISO_8859_3|ENC_ISO_8859_4|ENC_ISO_8859_5|ENC_ISO_8859_6|ENC_ISO_8859_7|ENC_ISO_8859_8|ENC_ISO_8859_9|ENC_KOI8_R,
Index: pango/fonts.c
===================================================================
RCS file: /cvs/gnome/pango/pango/fonts.c,v
retrieving revision 1.23
diff -u -r1.23 fonts.c
--- pango/fonts.c	2001/01/07 17:08:13	1.23
+++ pango/fonts.c	2001/06/09 22:36:30
@@ -27,6 +27,7 @@
 #include "pango-types.h"
 #include "pango-font.h"
 #include "pango-fontmap.h"
+#include "pango-utils.h"
 
 /**
  * pango_font_description_copy:
@@ -443,7 +444,7 @@
 /**
  * pango_font_get_coverage:
  * @font: a #PangoFont
- * @lang: the language tag (in the form of "en_US")
+ * @lang: the language tag (in the form of "en-us")
  * 
  * Compute the coverage map for a given font and language tag.
  * 
@@ -461,7 +462,7 @@
 /**
  * pango_font_find_shaper:
  * @font: a #PangoFont
- * @lang: the language tag (in the form of "en_US")
+ * @lang: the language tag (in the form of "en-us")
  * @ch: the ISO-10646 character code.
  * 
  * Find the best matching shaper for a font for a particular
@@ -530,7 +531,11 @@
 			const gchar      *lang,
 			PangoFontMetrics *metrics)
 {
+  const char *canon_lang;
+  
   g_return_if_fail (font != NULL);
 
-  PANGO_FONT_GET_CLASS (font)->get_metrics (font, lang, metrics);
+  canon_lang = lang ? pango_lang_canonicalize (lang) : NULL;
+  
+  PANGO_FONT_GET_CLASS (font)->get_metrics (font, canon_lang, metrics);
 }
Index: pango/modules.c
===================================================================
RCS file: /cvs/gnome/pango/pango/modules.c,v
retrieving revision 1.26
diff -u -r1.26 modules.c
--- pango/modules.c	2001/05/28 16:19:07	1.26
+++ pango/modules.c	2001/06/09 22:36:30
@@ -79,7 +79,8 @@
 /**
  * pango_find_map:
  * @lang: the language tag for which to find the map (in the form
- *        en or en_US)
+ *        en or en_US). This must be canonicalized by
+ *        pango_lang_canonicalize.
  * @engine_type_id: the render type for the map to find
  * @render_type_id: the engine type for the map to find
  * 
@@ -104,7 +105,7 @@
       if (map_info->engine_type_id == engine_type_id &&
 	  map_info->render_type_id == render_type_id)
 	{
-	  if (strcmp (map_info->lang, lang) == 0)
+	  if (map_info->lang == lang)
 	    break;
 	  else
 	    found_earlier = TRUE;
@@ -116,7 +117,7 @@
   if (!tmp_list)
     {
       map_info = g_new (PangoMapInfo, 1);
-      map_info->lang = g_strdup (lang);
+      map_info->lang = lang;
       map_info->engine_type_id = engine_type_id;
       map_info->render_type_id = render_type_id;
 
@@ -396,18 +397,11 @@
     {
       gchar **langs;
       gboolean is_exact = FALSE;
-      
+
       if (pair->info.ranges[i].langs)
 	{
-	  langs = g_strsplit (pair->info.ranges[i].langs, ";", -1);
-	  for (j=0; langs[j]; j++)
-	    if (strcmp (langs[j], "*") == 0 ||
-		strcmp (langs[j], info->lang) == 0)
-	      {
-		is_exact = TRUE;
-		break;
-	      }
-	  g_strfreev (langs);
+	  if (pango_lang_matches (info->lang, pair->info.ranges[i].langs))
+	    is_exact = TRUE;
 	}
       
       for (submap = pair->info.ranges[i].start / 256;
Index: pango/pango-attributes.c
===================================================================
RCS file: /cvs/gnome/pango/pango/pango-attributes.c,v
retrieving revision 1.26
diff -u -r1.26 pango-attributes.c
--- pango/pango-attributes.c	2001/04/17 01:49:07	1.26
+++ pango/pango-attributes.c	2001/06/09 22:36:30
@@ -21,7 +21,8 @@
 
 #include <string.h>
 
-#include <pango/pango-attributes.h>
+#include "pango-attributes.h"
+#include "pango-utils.h"
 
 struct _PangoAttrList
 {
@@ -159,49 +160,75 @@
 }
 
 /**
- * pango_attr_lang_new:
- * @lang: language tag (in the form "en_US")
+ * pango_attr_family_new:
+ * @family: the family or comma separated list of families
  * 
- * Create a new language tag attribute. 
+ * Create a new font family attribute.
  * 
  * Return value: the new #PangoAttribute.
  **/
 PangoAttribute *
-pango_attr_lang_new (const char *lang)
+pango_attr_family_new (const char *family)
 {
   static const PangoAttrClass klass = {
-    PANGO_ATTR_LANG,
+    PANGO_ATTR_FAMILY,
     pango_attr_string_copy,
     pango_attr_string_destroy,
     pango_attr_string_equal
   };
 
-  g_return_val_if_fail (lang != NULL, NULL);
+  g_return_val_if_fail (family != NULL, NULL);
   
-  return pango_attr_string_new (&klass, lang);
+  return pango_attr_string_new (&klass, family);
+}
+
+static PangoAttribute *
+pango_attr_lang_copy (const PangoAttribute *attr)
+{
+  return g_memdup (attr, sizeof (PangoAttrString));
+}
+
+static void
+pango_attr_lang_destroy (PangoAttribute *attr)
+{
+  g_free (attr);
 }
 
+static gboolean
+pango_attr_lang_equal (const PangoAttribute *attr1,
+			 const PangoAttribute *attr2)
+{
+  return ((PangoAttrString *)attr1)->value == ((PangoAttrString *)attr2)->value;
+}
+
 /**
- * pango_attr_family_new:
- * @family: the family or comma separated list of families
+ * pango_attr_lang_new:
+ * @lang: language tag (in the form "en_US")
  * 
- * Create a new font family attribute.
+ * Create a new language tag attribute. 
  * 
  * Return value: the new #PangoAttribute.
  **/
 PangoAttribute *
-pango_attr_family_new (const char *family)
+pango_attr_lang_new (const char *lang)
 {
+  PangoAttrString *result;
+  
   static const PangoAttrClass klass = {
-    PANGO_ATTR_FAMILY,
-    pango_attr_string_copy,
-    pango_attr_string_destroy,
-    pango_attr_string_equal
+    PANGO_ATTR_LANG,
+    pango_attr_lang_copy,
+    pango_attr_lang_destroy,
+    pango_attr_lang_equal
   };
 
-  g_return_val_if_fail (family != NULL, NULL);
+  g_return_val_if_fail (lang != NULL, NULL);
   
-  return pango_attr_string_new (&klass, family);
+  result = g_new (PangoAttrString, 1);
+
+  result->attr.klass = &klass;
+  result->value = (char *)pango_lang_canonicalize (lang);
+
+  return (PangoAttribute *)result;
 }
 
 static PangoAttribute *
@@ -1346,18 +1373,21 @@
  *           an attribute in the #PangoAttrList associated with the structure,
  *           or with @base. If you want to save this value, you should
  *           allocate it on the stack and then use pango_font_description_copy().
+ * @lang: if non-%NULl, location to store language tag for item, or %NULL
+ *        if non is found.
  * @extra_attrs: if non-%NULL, location in which to store a list of non-font
  *           attributes at the the current position; only the highest priority
  *           value of each attribute will be added to this list. In order
  *           to free this value, you must call pango_attribute_destroy() on
  *           each member.
  *
- * Get the font 
+ * Get the font and other attributes at the current iterator position.
  **/
 void
 pango_attr_iterator_get_font (PangoAttrIterator     *iterator,
 			      PangoFontDescription  *base,
 			      PangoFontDescription  *current,
+			      const char           **lang,
 			      GSList               **extra_attrs)
 {
   GList *tmp_list1;
@@ -1369,6 +1399,7 @@
   gboolean have_weight = FALSE;
   gboolean have_stretch = FALSE;
   gboolean have_size = FALSE;
+  gboolean have_lang = FALSE;
 
   g_return_if_fail (iterator != NULL);
   g_return_if_fail (base != NULL);
@@ -1376,6 +1407,9 @@
   
   *current = *base;
 
+  if (lang)
+    *lang = NULL;
+  
   if (extra_attrs)
     *extra_attrs = NULL;
   
@@ -1471,7 +1505,16 @@
 	      current->size = ((PangoAttrFloat *)attr)->value * base->size;
 	    }
           break;
-          
+	case PANGO_ATTR_LANG:
+	  if (lang)
+	    {
+	      if (!have_lang)
+		{
+		  have_lang = TRUE;
+		  *lang = ((PangoAttrString *)attr)->value;
+		}
+	    }
+	  break;
 	default:
 	  if (extra_attrs)
 	    {
Index: pango/pango-attributes.h
===================================================================
RCS file: /cvs/gnome/pango/pango/pango-attributes.h,v
retrieving revision 1.15
diff -u -r1.15 pango-attributes.h
--- pango/pango-attributes.h	2001/02/28 17:39:36	1.15
+++ pango/pango-attributes.h	2001/06/09 22:36:30
@@ -197,6 +197,7 @@
 void               pango_attr_iterator_get_font (PangoAttrIterator     *iterator,
                                                  PangoFontDescription  *base,
                                                  PangoFontDescription  *current,
+						 const char           **lang,
                                                  GSList               **extra_attrs);
 
 
Index: pango/pango-context.c
===================================================================
RCS file: /cvs/gnome/pango/pango/pango-context.c,v
retrieving revision 1.36
diff -u -r1.36 pango-context.c
--- pango/pango-context.c	2001/03/07 14:33:53	1.36
+++ pango/pango-context.c	2001/06/09 22:36:30
@@ -31,7 +31,7 @@
 {
   GObject parent_instance;
 
-  char *lang;
+  const char *lang;
   PangoDirection base_dir;
   PangoFontDescription *font_desc;
 
@@ -51,10 +51,7 @@
 			 PangoAttrList     *attrs,
                          PangoAttrIterator *cached_iter,
                          gint               n_chars,
-			 PangoEngineShape **shape_engines,
-			 PangoEngineLang  **lang_engines,
-			 PangoFont        **fonts,
-			 GSList           **extra_attr_lists);
+			 PangoAnalysis     *analyses);
 
 static void pango_context_init        (PangoContext      *context);
 static void pango_context_class_init  (PangoContextClass *klass);
@@ -126,9 +123,6 @@
 
   context = PANGO_CONTEXT (object);
 
-  if (context->lang)
-    g_free (context->lang);
-  
   g_slist_foreach (context->font_maps, (GFunc)g_object_unref, NULL);
   g_slist_free (context->font_maps);
 
@@ -440,11 +434,8 @@
 			const char   *lang)
 {
   g_return_if_fail (context != NULL);
-
-  if (context->lang)
-    g_free (context->lang);
 
-  context->lang = g_strdup (lang);
+  context->lang = pango_lang_canonicalize (lang);
 }
 
 /**
@@ -453,14 +444,15 @@
  * 
  * Retrieves the global language tag for the context.
  * 
- * Return value: the global language tag. This value must be freed with g_free().
+ * Return value: the global language tag. This value should
+ *   not be freed.
  **/
-char *
+G_CONST_RETURN char *
 pango_context_get_lang (PangoContext *context)
 {
   g_return_val_if_fail (context != NULL, NULL);
 
-  return g_strdup (context->lang);
+  return context->lang;
 }
 
 /**
@@ -536,12 +528,9 @@
   const char *p;
   const char *next;
   GList *result = NULL;
-  
-  PangoEngineShape **shape_engines;
-  PangoEngineLang  **lang_engines;
-  GSList           **extra_attr_lists;
-  PangoFont        **fonts;
 
+  PangoAnalysis *analyses;
+
   g_return_val_if_fail (context != NULL, NULL);
   g_return_val_if_fail (start_index >= 0, NULL);
   g_return_val_if_fail (length >= 0, NULL);
@@ -565,14 +554,11 @@
   pango_log2vis_get_embedding_levels (text_ucs4, n_chars, &base_dir,
                                       embedding_levels);
 
-  /* Storing these as ranges would be a lot more efficient,
-   * but also more complicated... we take the simple
-   * approach for now.
+  /* Storing ranges would be more efficient, but also more
+   * complicated... we take the simple approach for now.
    */
-  shape_engines = g_new0 (PangoEngineShape *, n_chars);
-  lang_engines = g_new0 (PangoEngineLang *, n_chars);
-  fonts = g_new0 (PangoFont *, n_chars);
-  extra_attr_lists = g_new0 (GSList *, n_chars);
+
+  analyses = g_new0 (PangoAnalysis, n_chars);
 
   /* Now, fill in the appropriate shapers, language engines and fonts for
    * each character.
@@ -581,8 +567,7 @@
   add_engines (context, text, start_index, length, attrs,
                cached_iter,
                n_chars,
-               shape_engines, lang_engines, fonts,
-	       extra_attr_lists);
+	       analyses);
 
   /* Make a GList of PangoItems out of the above results
    */
@@ -591,15 +576,19 @@
   p = text + start_index;
   for (i=0; i<n_chars; i++)
     {
+      PangoAnalysis *analysis = &analyses[i];
+      PangoAnalysis *last_analysis = i > 0 ? &analyses[i-1] : 0;
+      
       next = g_utf8_next_char (p);
       
       if (i == 0 ||
 	  text_ucs4[i] == '\t' || text_ucs4[i-1] == '\t' ||
 	  embedding_levels[i] != embedding_levels[i-1] ||
-	  shape_engines[i] != shape_engines[i-1] ||
-	  lang_engines[i] != lang_engines[i-1] ||
-	  fonts[i] != fonts[i-1] ||
-	  extra_attr_lists[i] != extra_attr_lists[i-1])
+	  analysis->shape_engine != last_analysis->shape_engine ||
+	  analysis->lang_engine != last_analysis->lang_engine ||
+	  analysis->font != last_analysis->font ||
+	  (analysis->lang != last_analysis->lang && strcmp (analysis->lang, last_analysis->lang) != 0) ||
+	  analysis->extra_attrs != last_analysis->extra_attrs)
 	{
           /* assert that previous item got at least one char */
           g_assert (item == NULL || item->length > 0);
@@ -610,15 +599,16 @@
 	  item->num_chars = 0;
 	  item->analysis.level = embedding_levels[i];
 	  
-	  item->analysis.shape_engine = shape_engines[i];
-	  item->analysis.lang_engine = lang_engines[i];
+	  item->analysis.shape_engine = analysis->shape_engine;
+	  item->analysis.lang_engine = analysis->lang_engine;
 
-	  item->analysis.font = fonts[i];
+	  item->analysis.font = analysis->font;
+	  item->analysis.lang = analysis->lang;
 
 	  /* Copy the extra attribute list if necessary */
-	  if (extra_attr_lists[i] && i != 0 && extra_attr_lists[i] == extra_attr_lists[i-1])
+	  if (analysis->extra_attrs && i != 0 && analysis->extra_attrs == last_analysis->extra_attrs)
 	    {
-	      GSList *tmp_list = extra_attr_lists[i];
+	      GSList *tmp_list = analysis->extra_attrs;
 	      GSList *new_list = NULL;
 	      while (tmp_list)
 		{
@@ -626,27 +616,23 @@
 					      pango_attribute_copy (tmp_list->data));
 		  tmp_list = tmp_list->next;
 		}
-	      item->extra_attrs = g_slist_reverse (new_list);
+	      item->analysis.extra_attrs = g_slist_reverse (new_list);
 	    }
 	  else
-	    item->extra_attrs = extra_attr_lists[i];
+	    item->analysis.extra_attrs = analysis->extra_attrs;
 
 	  result = g_list_prepend (result, item);
 	}
       else
-	g_object_unref (G_OBJECT (fonts[i]));
+	g_object_unref (analysis->font);
 
       item->length = (next - text) - item->offset;
       item->num_chars++;
       p = next;
     }  
 
+  g_free (analyses);
   g_free (embedding_levels);
-  g_free (shape_engines);
-  g_free (lang_engines);
-  g_free (fonts);
-  g_free (extra_attr_lists);
-  
   g_free (text_ucs4);
   
   return g_list_reverse (result);
@@ -738,7 +724,7 @@
     }
 
   if (result)
-    g_object_ref (G_OBJECT (result));
+    g_object_ref (result);
       
   return result;
 }
@@ -748,7 +734,7 @@
 
 static void
 load_font (PangoContext          *context,
-	   char                  *lang,
+	   const char            *lang,
 	   PangoFontDescription  *desc,
 	   PangoFont            **current_fonts,
 	   PangoCoverage        **current_coverages,
@@ -762,7 +748,7 @@
     {
       if (current_fonts[j])
 	{
-	  g_object_unref (G_OBJECT (current_fonts[j]));
+	  g_object_unref (current_fonts[j]);
 	  pango_coverage_unref (current_coverages[j]);
 	}
     }
@@ -876,13 +862,10 @@
 	     PangoAttrList     *attrs,
              PangoAttrIterator *cached_iter,
              gint               n_chars,
-	     PangoEngineShape **shape_engines,
-	     PangoEngineLang  **lang_engines,
-	     PangoFont        **fonts,
-	     GSList           **extra_attr_lists)
+	     PangoAnalysis     *analyses)
 {
   const char *pos;
-  char *lang = NULL;
+  const char *lang = NULL;
   int next_index;
   GSList *extra_attrs = NULL;
   PangoMap *lang_map = NULL;
@@ -891,7 +874,6 @@
   PangoFont *current_fonts[MAX_FAMILIES];
   PangoCoverage *current_coverages[MAX_FAMILIES];  
   PangoAttrIterator *iterator;
-  PangoAttribute *attr;
   gboolean first_iteration = TRUE;
   gunichar wc;
   int i = 0, j;
@@ -908,9 +890,11 @@
   pos = text + start_index;
   for (i=0; i<n_chars; i++)
     {
+      PangoAnalysis *analysis = &analyses[i];
+      
       if (first_iteration || pos - text == next_index)
 	{
-	  char *next_lang;
+	  const char *next_lang;
 	  PangoFontDescription next_desc;
 
           first_iteration = FALSE;
@@ -924,15 +908,13 @@
               pango_attr_iterator_range (iterator, NULL, &next_index);
             }
           
-	  attr = pango_attr_iterator_get (iterator, PANGO_ATTR_LANG);
-	  if (attr)
-	    next_lang = ((PangoAttrString *)attr)->value;
-	  else
+	  pango_attr_iterator_get_font (iterator, context->font_desc,
+                                        &next_desc, &next_lang, &extra_attrs);
+
+          if (!next_lang)
 	    next_lang = context->lang;
 
-	  if (i == 0 ||
-	      (lang != next_lang &&
-	       (lang == NULL || next_lang == NULL || strcmp (lang, next_lang) != 0)))
+	  if (i == 0 || lang != next_lang)
 	    {
 	      static guint engine_type_id = 0;
 	      static guint render_type_id = 0;
@@ -949,9 +931,6 @@
 					 engine_type_id, render_type_id);
 	    }
 
-	  pango_attr_iterator_get_font (iterator, context->font_desc,
-                                        &next_desc, &extra_attrs);
-          
 	  if (i == 0 ||
 	      !pango_font_description_equal (&current_desc, &next_desc))
 	    {
@@ -964,20 +943,21 @@
 
       wc = g_utf8_get_char (pos);
       pos = g_utf8_next_char (pos);
-	  
-      lang_engines[i] = (PangoEngineLang *)pango_map_get_engine (lang_map, wc);
-      fonts[i] = get_font (current_fonts, current_coverages, n_families, wc);
-
+      
+      analysis->lang_engine = (PangoEngineLang *)pango_map_get_engine (lang_map, wc);
+      analysis->font = get_font (current_fonts, current_coverages, n_families, wc);
+      analysis->lang = lang;
+      
       /* FIXME: handle reference counting properly on the shapers */
-      if (fonts[i])
-	shape_engines[i] = pango_font_find_shaper (fonts[i], lang, wc);
+      if (analysis->font)
+	analysis->shape_engine = pango_font_find_shaper (analysis->font, lang, wc);
       else
-	shape_engines[i] = NULL;
-
-      if (shape_engines[i] == NULL)
-        shape_engines[i] = &fallback_shaper;
+	analysis->shape_engine = NULL;
+      
+      if (analysis->shape_engine == NULL)
+        analysis->shape_engine = &fallback_shaper;
       
-      extra_attr_lists[i] = extra_attrs;
+      analysis->extra_attrs = extra_attrs;
     }
 
   g_assert (pos - text == start_index + length);
@@ -986,7 +966,7 @@
     {
       if (current_fonts[j])
 	{
-	  g_object_unref (G_OBJECT (current_fonts[j]));
+	  g_object_unref (current_fonts[j]);
 	  pango_coverage_unref (current_coverages[j]);
 	}
     }
Index: pango/pango-context.h
===================================================================
RCS file: /cvs/gnome/pango/pango/pango-context.h,v
retrieving revision 1.11
diff -u -r1.11 pango-context.h
--- pango/pango-context.h	2000/11/01 06:46:29	1.11
+++ pango/pango-context.h	2001/06/09 22:36:30
@@ -65,7 +65,7 @@
 void                  pango_context_set_font_description (PangoContext               *context,
 							  const PangoFontDescription *desc);
 PangoFontDescription *pango_context_get_font_description (PangoContext               *context);
-char *                pango_context_get_lang             (PangoContext               *context);
+G_CONST_RETURN char * pango_context_get_lang             (PangoContext               *context);
 void                  pango_context_set_lang             (PangoContext               *context,
 							  const char                 *lang);
 void                  pango_context_set_base_dir         (PangoContext               *context,
Index: pango/pango-item.c
===================================================================
RCS file: /cvs/gnome/pango/pango/pango-item.c,v
retrieving revision 1.4
diff -u -r1.4 pango-item.c
--- pango/pango-item.c	2001/01/23 15:15:04	1.4
+++ pango/pango-item.c	2001/06/09 22:36:30
@@ -55,18 +55,18 @@
   result->length = item->length;
   result->num_chars = item->num_chars;
 
+  result->analysis = item->analysis;
+  g_object_ref (result->analysis.font);
+  
   extra_attrs = NULL;
-  tmp_list = item->extra_attrs;
+  tmp_list = item->analysis.extra_attrs;
   while (tmp_list)
     {
       extra_attrs = g_slist_prepend (extra_attrs, pango_attribute_copy (tmp_list->data));
       tmp_list = tmp_list->next;
     }
 
-  result->extra_attrs = g_slist_reverse (extra_attrs);
-	  
-  result->analysis = item->analysis;
-  g_object_ref (G_OBJECT (result->analysis.font));
+  result->analysis.extra_attrs = g_slist_reverse (extra_attrs);
 
   return result;
 }
@@ -80,13 +80,13 @@
 void
 pango_item_free (PangoItem *item)
 {
-  if (item->extra_attrs)
+  if (item->analysis.extra_attrs)
     {
-      g_slist_foreach (item->extra_attrs, (GFunc)pango_attribute_destroy, NULL);
-      g_slist_free (item->extra_attrs);
+      g_slist_foreach (item->analysis.extra_attrs, (GFunc)pango_attribute_destroy, NULL);
+      g_slist_free (item->analysis.extra_attrs);
     }
-  
-  g_object_unref (G_OBJECT (item->analysis.font));
+
+  g_object_unref (item->analysis.font);
 
   g_free (item);
 }
Index: pango/pango-item.h
===================================================================
RCS file: /cvs/gnome/pango/pango/pango-item.h,v
retrieving revision 1.3
diff -u -r1.3 pango-item.h
--- pango/pango-item.h	2000/12/02 07:49:55	1.3
+++ pango/pango-item.h	2001/06/09 22:36:30
@@ -36,8 +36,10 @@
 {
   PangoEngineShape *shape_engine;
   PangoEngineLang  *lang_engine;
-  PangoFont        *font;
+  PangoFont *font;
   guint8 level;
+  const char *lang;
+  GSList *extra_attrs;
 };
 
 struct _PangoItem
@@ -45,7 +47,6 @@
   gint offset;
   gint length;
   gint num_chars;
-  GSList *extra_attrs;
   PangoAnalysis analysis;
 };
 
Index: pango/pango-layout.c
===================================================================
RCS file: /cvs/gnome/pango/pango/pango-layout.c,v
retrieving revision 1.65
diff -u -r1.65 pango-layout.c
--- pango/pango-layout.c	2001/06/08 16:02:59	1.65
+++ pango/pango-layout.c	2001/06/09 22:36:30
@@ -2274,20 +2274,29 @@
       PangoAttrList *tmp_attrs;
       PangoAttrIterator *iter;
       PangoFontDescription font_desc;
+      const char *lang;
       int i;
 
       layout_attrs = pango_layout_get_effective_attributes (layout);
       iter = pango_attr_list_get_iterator (layout_attrs);
       pango_attr_iterator_get_font (iter, pango_context_get_font_description (layout->context),
-				    &font_desc, NULL);
+				    &font_desc, &lang, NULL);
       
       tmp_attrs = pango_attr_list_new ();
+
       attr = pango_attr_font_desc_new (&font_desc);
       attr->start_index = 0;
       attr->end_index = 1;
-	  
       pango_attr_list_insert_before (tmp_attrs, attr);
       
+      if (lang)
+	{
+	  attr = pango_attr_lang_new (lang);
+	  attr->start_index = 0;
+	  attr->end_index = 1;
+	  pango_attr_list_insert_before (tmp_attrs, attr);
+	}
+	  
       items = pango_itemize (layout->context, " ", 0, 1, tmp_attrs, NULL);
 
       pango_attr_iterator_destroy (iter);
@@ -3345,6 +3354,7 @@
 		  pango_attr_iterator_get_font (iter,
 						base_font_desc,
 						&font_desc,
+						NULL,
 						NULL);
 		  break;
 		}
@@ -3365,9 +3375,9 @@
       font = pango_context_load_font (layout->context, &font_desc);
       if (font)
 	{
-	  char *lang = pango_context_get_lang (layout->context);
-	  pango_font_get_metrics (font, lang, &metrics);
-	  g_free (lang);
+	  pango_font_get_metrics (font,
+				  pango_context_get_lang (layout->context),
+				  &metrics);
 
 	  logical_rect->y = - metrics.ascent;
 	  logical_rect->height = metrics.ascent + metrics.descent;
@@ -3701,7 +3711,7 @@
 				  PangoRectangle *logical_rect,
 				  gboolean       *shape_set)
 {
-  GSList *tmp_list = item->extra_attrs;
+  GSList *tmp_list = item->analysis.extra_attrs;
 
   if (shape_set)
     *shape_set = FALSE;
Index: pango/pango-utils.c
===================================================================
RCS file: /cvs/gnome/pango/pango/pango-utils.c,v
retrieving revision 1.16
diff -u -r1.16 pango-utils.c
--- pango/pango-utils.c	2001/02/21 04:09:17	1.16
+++ pango/pango-utils.c	2001/06/09 22:36:30
@@ -570,7 +570,8 @@
   if (!config_hash)
     {
       char *filename;
-      char *home;
+      const char *home;
+      const char *envvar;
       
       config_hash = g_hash_table_new (g_str_hash, g_str_equal);
       filename = g_strconcat (pango_get_sysconf_subdirectory (),
@@ -589,9 +590,9 @@
 	  g_free (filename);
 	}
 
-      filename = g_getenv ("PANGO_RC_FILE");
-      if (filename)
-	read_config_file (filename, TRUE);
+      envvar = g_getenv ("PANGO_RC_FILE");
+      if (envvar)
+	read_config_file (envvar, TRUE);
     }
 }
 
@@ -885,6 +886,136 @@
 
   if (warn)
     g_warning ("Stretch must be ultra_condensed, extra_condensed, condensed, semi_condensed, normal, semi_expanded, expanded, extra_expanded, or ultra_expanded");
+  return FALSE;
+}
+
+static const char canon_map[256] = {
+   0,   0,   0,   0,   0,   0,   0,   0,    0,   0,   0,   0,   0,   0,   0,   0, 
+   0,   0,   0,   0,   0,   0,   0,   0,    0,   0,   0,   0,   0,   0,   0,   0, 
+   0,   0,   0,   0,   0,   0,   0,   0,    0,   0,   0,   0,   0,  '-',  0,   0, 
+   0,   0,   0,   0,   0,   0,   0,   0,    0,   0,   0,   0,   0,   0,   0,   0, 
+   0,  'a', 'b', 'c', 'd', 'e', 'f', 'g',  'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o',
+  'p', 'q', 'r', 's', 't', 'u', 'v', 'w',  'x', 'y', 'z',  0,   0,   0,   0,  '-',
+   0,  'a', 'b', 'c', 'd', 'e', 'f', 'g',  'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o',
+  'p', 'q', 'r', 's', 't', 'u', 'v', 'w',  'x', 'y', 'z',  0,   0,   0,   0,   0
+};
+   
+static gboolean
+lang_equal (gconstpointer v1,
+	    gconstpointer v2)
+{
+  const guchar *p1 = v1;
+  const guchar *p2 = v2;
+
+  while (*p2)
+    {
+      guchar value = canon_map[*p2];
+      if (value && value != *p1++)
+	return FALSE;
+      p2++;
+    }
+
+  return (*p1 == '\0');
+}
+
+static guint
+lang_hash (gconstpointer key)
+{
+  const guchar *p = key;
+  guint h = 0;
+  while (*p)
+    {
+      guchar value = canon_map[*p];
+      if (value)
+	h = (h << 5) - h + value;
+      p++;
+    }
+
+  return h;
+}
+
+/**
+ * pango_lang_canonicalize:
+ * @lang: a string representing a language tag
+ * 
+ * Take a RFC-3066 format language tag and canonicalize it
+ * by converting it to lowercase, mapping '_' to '-', and stripping
+ * all characters other than letters and '-'. Then the result
+ * is looked up in a central database, and a canonical value
+ * for the result is returned, that can be compared for equality
+ * with the results of other calls to pango_lang_canonicalize()
+ * 
+ * Return value: the canonical form of the language tag. 
+ **/
+G_CONST_RETURN char *
+pango_lang_canonicalize (const char *lang)
+{
+  static GHashTable *hash = NULL;
+  char *result;
+  int len;
+  char *p;
+
+  if (!hash)
+    hash = g_hash_table_new (lang_hash, lang_equal);
+
+  result = g_hash_table_lookup (hash, lang);
+  if (result)
+    return result;
+
+  len = strlen (lang);
+  result = g_malloc (len + 1);
+
+  p = result;
+  while (*lang)
+    {
+      char value = canon_map[*(guchar *)lang++];
+      if (value)
+	*(p++) = value;
+    }
+  *p++ = '\0';
+
+  g_hash_table_insert (hash, result, result);
+
+  return result;
+}
+
+/**
+ * pango_lang_matches:
+ * @lang: a RFC 3066 language tag, canonicalized as by pango_lang_canonicalize()
+ * @range_list: a list of language ranges, separated by ';' characters.
+ *   each element must either be '*', or a RFC 3066 language range
+ *   canonicalized as by pango_lang_canonicalize().
+ * 
+ * Checks if a language tag matches one of the elements in a list of
+ * language ranges. A language tag is considered to match a range
+ * in the list if the range is '*', the range is exactly the tag,
+ * or the range is a prefix of the tag, and the character after the
+ * tag is '-'.
+ **/
+gboolean
+pango_lang_matches (const char *lang,
+		    const char *range_list)
+{
+  const char *p = range_list;
+  gboolean done = FALSE;
+
+  while (!done)
+    {
+      const char *end = strchr (p, ';');
+      if (!end)
+	{
+	  end = p + strlen (p);
+	  done = TRUE;
+	}
+
+      if (strncmp (p, "*", 1) == 0 ||
+	  (strncmp (lang, p, end - p) == 0 &&
+	   (lang[end - p] == '\0' || lang[end - p] == '-')))
+	return TRUE;
+
+      p = end;
+    }
+
   return FALSE;
 }
 
Index: pango/pango-utils.h
===================================================================
RCS file: /cvs/gnome/pango/pango/pango-utils.h,v
retrieving revision 1.10
diff -u -r1.10 pango-utils.h
--- pango/pango-utils.h	2001/04/16 22:42:33	1.10
+++ pango/pango-utils.h	2001/06/09 22:36:30
@@ -73,6 +73,9 @@
  */
 G_CONST_RETURN char *   pango_get_lib_subdirectory (void);
 
+G_CONST_RETURN char *pango_lang_canonicalize (const char *lang);
+gboolean      pango_lang_matches  (const char *lang,
+					  const char *range_list);
 
 /* A couple of routines from fribidi that we either wrap or
  * provide ourselves.
@@ -82,4 +85,5 @@
 					     PangoDirection *pbase_dir,
 					     guint8         *embedding_level_list);
 gboolean pango_get_mirror_char              (gunichar        ch,
+
 					     gunichar       *mirrored_ch);
Index: pango/pangoft2.c
===================================================================
RCS file: /cvs/gnome/pango/pango/pangoft2.c,v
retrieving revision 1.17
diff -u -r1.17 pangoft2.c
--- pango/pangoft2.c	2001/05/04 21:30:53	1.17
+++ pango/pangoft2.c	2001/06/09 22:36:30
@@ -673,7 +673,7 @@
 			    const gchar      *lang,
 			    PangoFontMetrics *metrics)
 {
-  PangoFT2MetricsInfo *info;
+  PangoFT2MetricsInfo *info = NULL; /* Quiet GCC */
   PangoFT2Font *ft2font = (PangoFT2Font *)font;
   GSList *tmp_list;
       
@@ -1107,7 +1107,7 @@
 			       PangoAttrColor *bg_color,
 			       gboolean       *bg_set)
 {
-  GSList *tmp_list = item->extra_attrs;
+  GSList *tmp_list = item->analysis.extra_attrs;
 
   if (fg_set)
     *fg_set = FALSE;
Index: pango/pangox.c
===================================================================
RCS file: /cvs/gnome/pango/pango/pangox.c,v
retrieving revision 1.59
diff -u -r1.59 pangox.c
--- pango/pangox.c	2001/05/28 16:19:07	1.59
+++ pango/pangox.c	2001/06/09 22:36:30
@@ -824,7 +824,9 @@
 	  analysis.shape_engine = last_shaper;
 	  analysis.lang_engine = NULL;
 	  analysis.font = font;
+	  analysis.lang = lang;
 	  analysis.level = last_level;
+	  analysis.extra_attrs = NULL;
 	  
 	  pango_shape (start, p - start, &analysis, glyph_str);
 
@@ -1675,7 +1677,7 @@
 			     PangoAttrColor *bg_color,
 			     gboolean       *bg_set)
 {
-  GSList *tmp_list = item->extra_attrs;
+  GSList *tmp_list = item->analysis.extra_attrs;
 
   if (fg_set)
     *fg_set = FALSE;
? gdk-pixbuf-2.0-uninstalled.pc
? gdk-2.0-uninstalled.pc
? gtk+-2.0-uninstalled.pc
? langtag.diff
? docs/reference/gdk/gdk-undocumented.txt
? docs/reference/gdk-pixbuf/gdk-pixbuf-undocumented.txt
? docs/reference/gtk/gtk-undocumented.txt
? gdk-pixbuf/test-loaders
? m4macros/Makefile.in
? m4macros/Makefile
Index: docs/reference/gtk/tmpl/gtkrc.sgml
===================================================================
RCS file: /cvs/gnome/gtk+/docs/reference/gtk/tmpl/gtkrc.sgml,v
retrieving revision 1.32
diff -u -r1.32 gtkrc.sgml
--- docs/reference/gtk/tmpl/gtkrc.sgml	2001/06/08 19:03:22	1.32
+++ docs/reference/gtk/tmpl/gtkrc.sgml	2001/06/09 22:36:38
@@ -495,7 +495,6 @@
 #GtkRcStyle structures to form a #GtkStyle.
 </para>
 
- parent_instance: 
 @name: 
 @bg_pixmap_name: 
 @font_desc: 
Index: gdk/gdkpango.c
===================================================================
RCS file: /cvs/gnome/gtk+/gdk/gdkpango.c,v
retrieving revision 1.15
diff -u -r1.15 gdkpango.c
--- gdk/gdkpango.c	2001/05/11 23:29:37	1.15
+++ gdk/gdkpango.c	2001/06/09 22:36:38
@@ -452,7 +452,7 @@
 			       PangoRectangle *ink_rect,
 			       PangoRectangle *logical_rect)
 {
-  GSList *tmp_list = item->extra_attrs;
+  GSList *tmp_list = item->analysis.extra_attrs;
 
   if (strikethrough)
       *strikethrough = FALSE;
Index: gtk/gtkclist.c
===================================================================
RCS file: /cvs/gnome/gtk+/gtk/gtkclist.c,v
retrieving revision 1.195
diff -u -r1.195 gtkclist.c
--- gtk/gtkclist.c	2001/06/08 18:09:33	1.195
+++ gtk/gtkclist.c	2001/06/09 22:36:38
@@ -3027,11 +3027,10 @@
       PangoContext *context = gtk_widget_get_pango_context (widget);
       PangoFontMetrics metrics;
       PangoFont *font = pango_context_load_font (context, widget->style->font_desc);
-      gchar *lang = pango_context_get_lang (context);
+      const gchar *lang = pango_context_get_lang (context);
 
       pango_font_get_metrics (font, lang, &metrics);
       
-      g_free (lang);
       g_object_unref (G_OBJECT (font));
       
       if (!GTK_CLIST_ROW_HEIGHT_SET(clist))
Index: gtk/gtkentry.c
===================================================================
RCS file: /cvs/gnome/gtk+/gtk/gtkentry.c,v
retrieving revision 1.132
diff -u -r1.132 gtkentry.c
--- gtk/gtkentry.c	2001/06/08 16:06:59	1.132
+++ gtk/gtkentry.c	2001/06/09 22:36:38
@@ -1020,7 +1020,7 @@
   GtkEntry *entry;
   PangoFontMetrics metrics;
   PangoFont *font;
-  gchar *lang;
+  const gchar *lang;
   gint xborder, yborder;
   
   g_return_if_fail (widget != NULL);
@@ -1035,7 +1035,6 @@
 				  widget->style->font_desc);
   lang = pango_context_get_lang (gtk_widget_get_pango_context (widget));
   pango_font_get_metrics (font, lang, &metrics);
-  g_free (lang);
   
   g_object_unref (G_OBJECT (font));
 
Index: gtk/gtkspinbutton.c
===================================================================
RCS file: /cvs/gnome/gtk+/gtk/gtkspinbutton.c,v
retrieving revision 1.62
diff -u -r1.62 gtkspinbutton.c
--- gtk/gtkspinbutton.c	2001/06/05 20:07:02	1.62
+++ gtk/gtkspinbutton.c	2001/06/09 22:36:38
@@ -569,7 +569,7 @@
     {
       PangoFontMetrics metrics;
       PangoFont *font;
-      gchar *lang;
+      const gchar *lang;
       gint width;
       gint w;
       int string_len;
@@ -579,7 +579,6 @@
                                       widget->style->font_desc);
       lang = pango_context_get_lang (gtk_widget_get_pango_context (widget));
       pango_font_get_metrics (font, lang, &metrics);
-      g_free (lang);
       g_object_unref (G_OBJECT (font));
       
       /* Get max of MIN_SPIN_BUTTON_WIDTH, size of upper, size of lower */
Index: gtk/gtkstyle.c
===================================================================
RCS file: /cvs/gnome/gtk+/gtk/gtkstyle.c,v
retrieving revision 1.66
diff -u -r1.66 gtkstyle.c
--- gtk/gtkstyle.c	2001/06/04 23:15:50	1.66
+++ gtk/gtkstyle.c	2001/06/09 22:36:38
@@ -4076,7 +4076,7 @@
 
       if (run)
         {
-          tmp_list = run->item->extra_attrs;
+          tmp_list = run->item->analysis.extra_attrs;
 
           while (tmp_list != NULL)
             {
Index: gtk/gtktextdisplay.c
===================================================================
RCS file: /cvs/gnome/gtk+/gtk/gtktextdisplay.c,v
retrieving revision 1.24
diff -u -r1.24 gtktextdisplay.c
--- gtk/gtktextdisplay.c	2001/05/18 03:41:30	1.24
+++ gtk/gtktextdisplay.c	2001/06/09 22:36:38
@@ -191,7 +191,7 @@
                    PangoRectangle *ink_rect,
                    PangoRectangle *logical_rect)
 {
-  GSList *tmp_list = run->item->extra_attrs;
+  GSList *tmp_list = run->item->analysis.extra_attrs;
     
   while (tmp_list)
     {
@@ -653,7 +653,7 @@
 get_item_properties (PangoItem          *item,
                      GtkTextAppearance **appearance)
 {
-  GSList *tmp_list = item->extra_attrs;
+  GSList *tmp_list = item->analysis.extra_attrs;
 
   *appearance = NULL;
 
Index: gtk/gtktextlayout.c
===================================================================
RCS file: /cvs/gnome/gtk+/gtk/gtktextlayout.c,v
retrieving revision 1.63
diff -u -r1.63 gtktextlayout.c
--- gtk/gtktextlayout.c	2001/06/08 16:07:00	1.63
+++ gtk/gtktextlayout.c	2001/06/09 22:36:38
@@ -1492,7 +1492,7 @@
 static gboolean
 is_shape (PangoLayoutRun *run)
 {
-  GSList *tmp_list = run->item->extra_attrs;
+  GSList *tmp_list = run->item->analysis.extra_attrs;
     
   while (tmp_list)
     {
@@ -1582,6 +1582,7 @@
       PangoAttribute *insert_attr;
       GSList *extra_attrs = NULL;
       GSList *tmp_list;
+      const char *lang;
       gint start, end;
 
       pango_attr_iterator_range (iter, &start, &end);
@@ -1590,7 +1591,7 @@
 	end = layout->preedit_len;
       
       pango_attr_iterator_get_font (iter, &style->font,
-				    &font_desc, &extra_attrs);
+				    &font_desc, &lang, &extra_attrs);
       
       tmp_list = extra_attrs;
       while (tmp_list)
@@ -1630,6 +1631,15 @@
       insert_attr->end_index = end + offset;
       
       pango_attr_list_insert (attrs, insert_attr);
+
+      if (lang)
+	{
+	  insert_attr = pango_attr_lang_new (lang);
+	  insert_attr->start_index = start + offset;
+	  insert_attr->end_index = end + offset;
+	  
+	  pango_attr_list_insert (attrs, insert_attr);
+	}
 
       add_generic_attrs (layout, &appearance, end - start,
                          attrs, start + offset,


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]