Re: State of gdk-pixbuf

On Fri, 2014-10-24 at 13:30 +0000, Benjamin Otte wrote:
Bastien Nocera <hadess <at>> writes:

I'm particularly interested to know what cairo, pixman and other image
manipulation libraries can do for us. Benjamin surely has comments[2] :)

I think gdk-pixbuf should just die already, but it won't. And now you're
resurrecting it, dammit!

While there's nothing to replace it, probably not. And so far, I'm
mainly wrangling patches and fixing small bugs.

The reason is that I think that gdk-pixbuf is both a rather old library so
it has very sucky conventions in lots of places (like exposing way too many
externals) and that it isn't a good fit for the use cases we have.
But it's ubiquitous and easy to use when it does work, so it's always the
place people go to.

Exposing too many internals? Or do you mean its API is too wide?

I'll try to elaborate on the use cases we have and why it's not been
replaced for each of those

1. image loading
gdk-pixbuf provides a generic way to hand it a file or stream and get pixels
out the other end. There is no other way to do this, unless you are a
masochist and want to use the image loading libs directly. The loaders in
gdk-pixbuf have also been sanitized well enough to be safe (enough) against
random data from the Internet.
The reason gdk-pixbuf is used here is because there's no replacement that is
as easy to use and integrates as well into the Gnome stack. Webkit has its
own loaders, but they are internal to Webkit (I'd expect them to be even
safer). Gimp/Gegl has its own loaders, but they pull in libgimp (or you need
to code your own widgets to display them).

OK, so we agree that we should be keeping that part at least. If I
remember correctly, fixing potential security problems in the loaders is
how Matthias got started ;)

We could probably do with a fuzzer and throwing images at it, or with a
couple of coverity runs.

2. holding pixel data
A GdkPixbuf is a kind of agnostic way to hold a (width, height, contents)
object. It's rather ad at this because it uses the worst format possible
(unpremultiplied RGBA in the wrong order). 
GTK is slowly replacing its internal APIs with cairo_surface_t here. And I
expect this to potentially change again to some sort of texture thing once
we move towards GL.

I've been looking at a way to make this more efficient, and babl might
be a good way to store decoded data in a format that's closer to the
format used internally. Eg. a grayscale picture shouldn't take as much
room in memory as a same-dimension colour one.

Once you actually want to display, you'd need to convert it to a format
cairo understands. We have some pixbuf/cairo APIs in GDK, should those
be in gdk-pixbuf instead?

3. an area of custom drawing
GTK kindly kindly provides lots of API to use GdkPixbuf everywhere - from
gdk_cairo_set_source_pixbuf() to lots of widgets that can display it
(GtkImage, GtkEntry, GtkClipboard, GdkCellRendererPixbuf...). So people use
pixbufs whenever they have some custom drawing that they want displayed
(like icons, custom avatar pictures or even graphs). There are various
limitations here - the most important one being the problems around resizes,
but it's the best thing we have.
One could use cairo_surface_t for this, but I'm not very happy with exposing
it, because cairo_surface_t has too many weird additional features, like
device scales, device offsets, it being Cairo backend-bound, its ability to
be changed at will and last but not least it's just not a GObject.
What you really want is some sort of object that has width, height and a
draw function and have lots of subclasses for pixbufs, gradients, single
color, whatever. But such an object doesn't exist. is what GTK
uses internally for the different image types CSS supports. You can look for
the various subclasses that implement those from url() and linear-gradient()
to -gtk-icontheme() and cross-fade().

gdk_cairo_surface_create_from_pixbuf() already exists. If we moved this
sort of call into gdk-pixbuf, it could know about the internals of the
storage, and convert it more efficiently, without an intermediate step.

4. animations
GdkPixbuf has rudimentary animation API. That API is very sucky from both
sides - the gif loader is as ugly at implementing it as it is inconvenient
to use.

What would this API look like ideally?

I think the GIF loader should use giflib instead of its own code, would
reduce the maintenance burden and probably make the code easier to read.

 Animations are separate from regular pixbufs, so you often need
multiple code paths if you want to support animations. But it's the only
thing we have and you don't have to think about it when passing a filename
to GtkImage, so it's seen various usages.
There's no good replacement. All solutions that really support animating
images are really complex. SVG requires full renderers (usually web
browsers), even rudimentary movies require a GStreamer-like framework and
simple animations (like displaying progressive image loads) just aren't done.


5. simple drawing operations
This is the only thing that has been replaced in all code that isn't
unmaintained or dusty because Cairo is just that neat to use, even if it's a
bit awkward to first. The only place where it is still used is where there's
operations that people used to do in 1990 that don't have a modern
equivalent and people still want to behave like those old apps.

Completely agree here.

So to reiterate, I wish 3 things would happen:
- graphic-heavy apps like eog or gthumb would go use Gegl/libgimp. They'd
get nice filters for free!

Problems being that:
- GEGL can only load from local files (no stream API, no incremental
loading), though it can load GdkPixbufs.
- GEGL uses gdk-pixbuf's loaders

So this hangs on to GdkPixbuf's storage not being very efficient. If we
used babl internally, we could probably short-circuit the conversion
from the internal/native format to GeglBuffer.

- there would be a nice library for "an area with width and height that can
draw itself", complete with implementations for drawing with cairo and image
loaders, icontheme and animation support and hooks that are convenient and
performant enough to use for GStreamer video implementation and resizable
animated SVGs.

Which would be something other than gdk-pixbuf.

- gdk--pixbuf to go away.

- we still want the loaders it provides
- we still need GdkPixbuf, the object

I think that it means that we should still be fixing the loaders if
there are bugs in them, and modifying GdkPixbuf's storage to be more

We could certainly nuke the rest if we think that other libraries would
be better equipped for resizing, clipping, scaling, and compositing.

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]