Re: State of gdk-pixbuf

[Benjamin Otte <otte gnome org>]

Bastien Nocera <hadess <at> hadess.net> writes:

> I'm particularly interested to know what cairo, pixman and other image
> manipulation libraries can do for us. Benjamin surely has comments[2] :)
I think gdk-pixbuf should just die already, but it won't. And now you're
resurrecting it, dammit!

The reason is that I think that gdk-pixbuf is both a rather old library so
it has very sucky conventions in lots of places (like exposing way too many
externals) and that it isn't a good fit for the use cases we have.
But it's ubiquitous and easy to use when it does work, so it's always the
place people go to.

I'll try to elaborate on the use cases we have and why it's not been
replaced for each of those

1. image loading
gdk-pixbuf provides a generic way to hand it a file or stream and get pixels
out the other end. There is no other way to do this, unless you are a
masochist and want to use the image loading libs directly. The loaders in
gdk-pixbuf have also been sanitized well enough to be safe (enough) against
random data from the Internet.
The reason gdk-pixbuf is used here is because there's no replacement that is
as easy to use and integrates as well into the Gnome stack. Webkit has its
own loaders, but they are internal to Webkit (I'd expect them to be even
safer). Gimp/Gegl has its own loaders, but they pull in libgimp (or you need
to code your own widgets to display them).

2. holding pixel data
A GdkPixbuf is a kind of agnostic way to hold a (width, height, contents)
object. It's rather ad at this because it uses the worst format possible
(unpremultiplied RGBA in the wrong order). 
GTK is slowly replacing its internal APIs with cairo_surface_t here. And I
expect this to potentially change again to some sort of texture thing once
we move towards GL.


3. an area of custom drawing
GTK kindly kindly provides lots of API to use GdkPixbuf everywhere - from
gdk_cairo_set_source_pixbuf() to lots of widgets that can display it
(GtkImage, GtkEntry, GtkClipboard, GdkCellRendererPixbuf...). So people use
pixbufs whenever they have some custom drawing that they want displayed
(like icons, custom avatar pictures or even graphs). There are various
limitations here - the most important one being the problems around resizes,
but it's the best thing we have.
One could use cairo_surface_t for this, but I'm not very happy with exposing
it, because cairo_surface_t has too many weird additional features, like
device scales, device offsets, it being Cairo backend-bound, its ability to
be changed at will and last but not least it's just not a GObject.
What you really want is some sort of object that has width, height and a
draw function and have lots of subclasses for pixbufs, gradients, single
color, whatever. But such an object doesn't exist.
https://git.gnome.org/browse/gtk+/tree/gtk/gtkcssimageprivate.h is what GTK
uses internally for the different image types CSS supports. You can look for
the various subclasses that implement those from url() and linear-gradient()
to -gtk-icontheme() and cross-fade().

4. animations
GdkPixbuf has rudimentary animation API. That API is very sucky from both
sides - the gif loader is as ugly at implementing it as it is inconvenient
to use. Animations are separate from regular pixbufs, so you often need
multiple code paths if you want to support animations. But it's the only
thing we have and you don't have to think about it when passing a filename
to GtkImage, so it's seen various usages.
There's no good replacement. All solutions that really support animating
images are really complex. SVG requires full renderers (usually web
browsers), even rudimentary movies require a GStreamer-like framework and
simple animations (like displaying progressive image loads) just aren't done.

5. simple drawing operations
This is the only thing that has been replaced in all code that isn't
unmaintained or dusty because Cairo is just that neat to use, even if it's a
bit awkward to first. The only place where it is still used is where there's
operations that people used to do in 1990 that don't have a modern
equivalent and people still want to behave like those old apps.

So to reiterate, I wish 3 things would happen:
- graphic-heavy apps like eog or gthumb would go use Gegl/libgimp. They'd
get nice filters for free!
- there would be a nice library for "an area with width and height that can
draw itself", complete with implementations for drawing with cairo and image
loaders, icontheme and animation support and hooks that are convenient and
performant enough to use for GStreamer video implementation and resizable
animated SVGs.
- gdk--pixbuf to go away.

Benjamin