Pango OS X pango_coverage_from_cf_charset issue

[Bill Kelly <billk cts com>]

Hi,

I'm finding that Pango, using the pango-basic-coretext.so
module on OS X, appears to be returning
PANGO_COVERAGE_NONE for higher unicode code points.

The string I'm testing with has two Japanese
characters at the end:

"_MG_9781.jpg:\u5927\u962A"

(Raw UTF-8 bytes are:
"_MG_9781.jpg:\xE5\xA4\xA7\xE9\x98\xAA" )


(It renders fine via libclutter/Pango on Win32, but
the Japanese glyphs are boxes on OS X.)


On OS X, the font Pango is working with is:

(gdb) p *priv->face
$40 = {
  parent_instance = {
    parent_instance = {
      g_type_instance = {
        g_class = 0x2809b50
      },
      ref_count = 1,
      qdata = 0x0
    }
  },
  family = 0x180b760,
  coverage = 0x201d890,
  postscript_name = 0xc4dbd0 "Helvetica",
  style_name = 0xc4f080 "Regular",
  weight = PANGO_WEIGHT_NORMAL,
  traits = 0,
  synthetic_italic = 0
}
(gdb) p *priv->desc
$41 = {
  family_name = 0xc4dd20 "Sans",
  style = PANGO_STYLE_NORMAL,
  variant = PANGO_VARIANT_NORMAL,
  weight = PANGO_WEIGHT_NORMAL,
  stretch = PANGO_STRETCH_NORMAL,
  gravity = PANGO_GRAVITY_SOUTH,
  mask = 47,
  static_family = 0,
  size_is_absolute = 0,
  size = 32768
}


However, the coverage blocks for these higher
codepoints end up with no data, defaulting to
PANGO_COVERAGE_NONE.

For example:

(gdb) p block_index
$28 = 150

(gdb) p coverage->blocks[block_index]
$31 = {
  data = 0x0,
  level = PANGO_COVERAGE_NONE
}



I'm using the latest Pango sources from git
(needed for the new PANGO_SYSCONFDIR env.
capability.)

I'm on OS X 10.6.7, and have built Pango as:

CXXFLAGS="-arch i386 -g -O0 -fno-rtti -fno-exceptions" CFLAGS="-arch i386 -g -O0"  \
 ./autogen.sh --prefix=/usr/local/clutter-i386 --enable-shared --enable-introspection=no --without-x


My application doesn't use Pango directly, but
rather indirectly via libclutter.

I'm new to Pango, and my investigation of the
issue has reached a point where I'm not sure
what to try next.

Here's where I got stuck:

The function deciding the font coverage is:

static PangoCoverage *
pango_coverage_from_cf_charset (CFCharacterSetRef charset)
{
  CFIndex i, length;
  CFDataRef bitmap;
  const UInt8 *ptr;
  PangoCoverage *coverage;

  coverage = pango_coverage_new ();

  bitmap = CFCharacterSetCreateBitmapRepresentation (kCFAllocatorDefault,
                                                     charset);

  /* We only handle the BMP plane */
  length = MIN (CFDataGetLength (bitmap), 8192);
  ptr = CFDataGetBytePtr (bitmap);

  /* FIXME: can and should this be done more efficiently? */
  for (i = 0; i < length; i++)
    {
      int j;

      for (j = 0; j < 8; j++)
        pango_coverage_set (coverage, i * 8 + j,
                            ((ptr[i] & (1 << j)) == (1 << j)) ?
                            PANGO_COVERAGE_EXACT : PANGO_COVERAGE_NONE);
    }

  CFRelease (bitmap);

  return coverage;
}


However, I've dumped out the bitmap returned by
CFCharacterSetCreateBitmapRepresentation, and
indeed, it's largely blank in the upper code
pages.

I had thought "Helvetica" should contain the
Japanese glyphs.

Would I need to specify the font in a special
way?


Thanks for any assistance,

Bill