GNOME.org
 

Re: An alternative to gdk-pixbuf

On Thu, 6 Sep 2018 11:39:59 +0100
Emmanuele Bassi <ebassi gmail com> wrote:


On Wed, 5 Sep 2018 at 19:25, Magnus Bergman
<magnus bergman snisurset net> wrote:


On Wed, 5 Sep 2018 17:28:22 +0100
Emmanuele Bassi <ebassi gmail com> wrote:
 

We're phasing out Cairo in favour of the CSS rendering model,
implemented on top of OpenGL and Vulkan, as it's the API that most
closely matches the requirements of GTK.  


I'm not sure I quite understand what you are saying. What does this
mean for image loading if terms of actual implementation? What would
ideally happen then GTK+ needs to load an image (because the
application called gtk_image_new_from_file() for example)?

 

We're still using GdkPixbuf, and for the 4.0 API series we still have
GdkPixbuf types exposed in the GTK API.

For future API series (5.x, 6.x, …) we may revisit this, with the
option of moving icon loading into GTK itself.


Okay, sound reasonable. 



Cairo is still used as a fallback mechanism — both when running on
platforms without support for OpenGL or Vulkan, and in the interim
period while the GL/Vulkan rendering pipelines inside GTK don't
support a CSS feature. Additionally, you may still use Cairo for
2D high quality rendering, for printing and for custom drawing.  


But then it comes to printing and custom drawing I'm on my own then
it comes to loading the images? Which is okey of course since
gdk-pixbuf is kind of a separate library already. But isn't it a
good thing to share common image loading code?
 


Not really; icons—especially ones that are identified via icon theme
names—are a very narrow subset of "all the possible images and
formats in human history".


Gegl is great for image editing. But not as much for simple viewing.


This is debatable. If I'm viewing a 4000x4000 RGB image on a hidpi
display I'm already pushing gdk-pixbuf and cairo to their limits
because of the scaling factor applied to the window — not only the
buffer gets loaded uncompressed to allow for zooming, but the image
viewer needs to render a CPU-scaled down copy of the image.


I often deal with images of those sizes and I haven't experienced any
problems with this myself (sure it's not as fast as viewing small
images). But I will look into it. I though cairo was capable of
utilising the rendering back end for it's scaling. I don't see why
cairo NEEDS to use CPU-scaling, so maybe it can be fixed.



It doesn't do animation  


Animated formats are a dying breed, and they have all but killed by
VP9 and new, web-friendly video formats. Social platforms will take a
GIF and turn it into a video transparently.

Additionally, GTK 4.x already has new API to render videos and other
frame-based media sources through GStreamer.


I must agree this sounds like a good decision. Even though I don't
think animated GIFs will die any time soon, it's not really anything
GTK+ should need to pay special attention to. From my perspective,
being interested in the art scene, animation might not be very common,
but formats supporting animation certainly are. It's not only in the
form of a series of frames (like GIF), it's also palette cycling, or
simply one blinking color I include in the concept of animation. I've
also seen image formats that shows the current time. Those, I think,
should ideally be displayed with the time updating. So that's also a
kind of animation. Does GStreamer support vector animations, by the way?



From the perspective of a consumer of GdkPixbuf, the only thing
that  
an image loading library should do is, pretty literally, load
images. No scaling, no compositing, no rendering. Saving to a
file may be interesting — but that opens the whole transcoding
can of worms, so maybe it should just be a debugging feature.
Ideally, I/O operations should happen in a separate thread, and
possible use multi-threading to chunk out the work on multiple
cores; it definitely needs to integrate with GIO, as it's a
modern API that well maps to GUIs.  


Yes, I very much agree with this. Except I think it has to do
rendering of some sort. If an image contains a triangle, it has to
be converted to a function call that renders it (using
OpenGL/Vulcan/cairo) somehow. 


That's up to the toolkit, as it's going to be the one deciding how to
render the rest of the UI. There's no way for you to know, from the
outside, how to efficiently render anything.

That's why it's much, much better to get a memory buffer with the
contents of the image, and let the toolkit pass it to the GPU.

Vector formats are different, but that's why we have API like Cairo or
Skia; I'm also waiting for web browsers to implement SVG rendering on
the GPU where possible, with the help of new primitives from the
GL/Vulkan implementation.


That means one code path for images that are certainly raster based and
another code path for images that could contain vector elements? My
thought was to always use cairo (or something similar) and just ignore
if the images are raster based or vector based (or a combination of the
two). As long as there is no need to directly access the pixels, there
is generally no need to even know difference then using the API. But you
say there is no way for cairo to know how to effectively render a
pixmap? Could this problem possibly be solved in cairo? I guess the
problem is that cairo can't keep track of things such as images stored
in the texture RAM. Would it be theoretically possible to expand cairos
API to support such things, you think? So that a PDF viewer for example
could store embedded images in texture RAM and render the whole thing
in different sizes super fast, by only using cairo (or at least
something like cairo).



In the near future, I'll very likely deprecate most of GdkPixbuf's  

API, except for the I/O operations; I'd also be happy to seal off
most of its internals, within the ABI stability promise, to avoid
leakage of internal state.  


Will the loader plugin API go away, you think?
 


No API will ever go away: there are no plans for a gdk-pixbuf-3.0. The
deprecations would mostly apply to API that is either long since been
replaced by Cairo (scale/composite), or that is weirdly ad hoc, like
saturate_and_pixelate(). Ideally, I'd like to deprecate the option to
build gdk-pixbuf without depending on GIO to do MIME type sniffing,
as GIO has been fixed to work on Windows and macOS, and it is a
required dependency anyway. The animation API is pretty much a
garbage fire, so it may go on the chopping block as well. Of course,
deprecated API will keep working as well—or as badly—as it works
today, so people can still use it. Moving pixbuf loaders to separate
processes, and wrap them in sandboxes, would be a fairly good thing
to do; it need to be decided at run time, though, because there are
many users of GdkPixbuf that already run in a sandbox, which prevents
creating smaller sandboxes inside it.


There was talk about ignoring the sniffing patterns (as well as the
file extensions) provided by the image loader plugins and only relying
on glib (and the shared MIME database) for sniffing the type. But it's
now a compile time option you say?

How does file type handling work on Windows nowadays? Last time I tried
it (about the time GTK+ 3,0 was released), it didn't work at all because
glib on Windows relied on Windows' own type system which is (or was)
completely broken. (I needed to ship the MIME database and the Unix
version of the type checking code from glib compiled for Windows with
my application in order to get it to work reliably.)

As I wrote in another answer, I don't think it's ideal to put small
tasks such as image decoding in micro sandboxes. It's probably better
to put even more data processing stuff in the same sandbox (like
applying filters to an image for example). Especially if the two
approaches conflicts with one another.
[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]