gnome-shell / mutter / ARB_texture_rectangle

[Owen Taylor <otaylor redhat com>]

So, I've been working on getting the beginnings of gnome-shell going
based on mutter (metacity-clutter.) And a large part of that has been
debugging mutter on various hardware.

The biggest problem so far is that ClutterGLXPixmapTexture has no
support for the ARB_texture_rectangle extension. So, it either uses 
ARB_texture_non_power_of_two, or it falls back to tiling.

Talking to Neil, he said that support for ARB_texture_rectangle used to
exist in clutter, but was removed because there was an assumption that
if EXT_texture_for_pixmap extension was present, then
ARB_texture_non_power_of_two would be present as well.

Unfortunately, that's not the case - in particular Radeon r3xx and r5xx
don't have full hardware support for NPOT textures; they are pretty
much restricted to supporting only ARB_texture_rectangle. On these
cards mutter performs poorly using the GetImage fallback.

In addition, there seem to be a wide variety of driver/hardware
combinations where compiz works but mutter badly misbehaves (anything
but the very latest Intel drivers, some versions of the nVidia
proprietary drivers, etc.) It's pure speculation that using
ARB_texture_rectangle would resolve these issues, but it's at least
something I want to try.

So, anyways, the conclusion is that I need ARB_texture_rectangle
support. I have a hacked up version of GLXPixmapTexture that does this
by either using cogl_texture or directly using a GL texture, but this
runs into some problems because mutter uses CloneTexture and otherwise
expects to be able to get the cogl_texture and draw with it.

I see two basic routes I could take:

1) I could add support for ARB_texture_rectangle into cogl_texture.
   Presumably this would look like cogl_texture_new_rectangle()
   or something ... you'd have to explicitly ask to get a rectangular
   texture, since they do behave significantly different.

   (Though more different than a sliced up texture? Not sure...)

   The main downside here, assuming I could get a change like this
   into clutter, would by tying gnome-shell to the devel branch
   of clutter.

2) I could cut and paste GLXPixmapTexture into the mutter sources, add
   API to it to reveal a) whether it is using texture_rectangle 
   b) to get the rectangular texture handle. Then modify 
   MutterShapedTexture and TidyTextureFrame to use that API.

   This would have the big advantage of being something I could do
   completely within my branch of metacity-compositor. But it is
   significantly messier, and doesn't help other people trying to
   use GLXPixmapTexture.

Advice or alternate suggestions appreciated.

- Owen