Re: An alternative to gdk-pixbuf

[Magnus Bergman <magnus bergman snisurset net>]

On Fri, 07 Sep 2018 12:51:32 +0200
Bastien Nocera <hadess hadess net> wrote:

> > > Gegl is great for image editing. But not as much for simple
> > > viewing.  
> >
> > This is debatable. If I'm viewing a 4000x4000 RGB image on a hidpi
> > display I'm already pushing gdk-pixbuf and cairo to their limits
> > because of the scaling factor applied to the window — not only the
> > buffer gets loaded uncompressed to allow for zooming, but the image
> > viewer needs to render a CPU-scaled down copy of the image.
> >
> > Sure, for viewing a 500x400px image macro for a meme we're fine; but
> > we're fine with gdk-pixbuf as well, so there's really no need to
> > change to a different image loading library.  
>
> I concur, 4000x4000 would likely OOM your machine, and it's not really
> fixable:
> https://gitlab.gnome.org/GNOME/gdk-pixbuf/issues/30
>
> Viewers should use GEGL, and GEGL should be taught about more formats.
> That still leaves many GIF files unhandled though, and I'm not sure
> we'd want apps writing their own GIF loader, seeing how complicated
> that is.

How do you mean it's not fixable? Of course it's not fixable that
processing larger amounts of data will require more processing power.
So I think it's kind of obvious that creating thumbnails for many large
images will take it's time. But that shouldn't need be too much of a
problem. Without looking too deeply into it, it looks like a problem
with nautilus to me. Obviously it's a bad idea to to run the
thumbnailers synchronously (which I can't imagine Nautilus does), and
it's also bad to run too many memory hungry tasks in parallel. But
that must be fixable, right? Like limiting the the total size in bytes
of files being allowed to be processed in parallel, for example.

At least now I see the problem with large images. I have no problem with
4000x4000, such images load instantly and zooms and pans smoothly. But
the 11000x14000 image was a bit sluggish to zoom and pan. With Gimp it
took a lot longer to load it, but panning and zooming was fast and
smooth. While gdk-pixbuf is optimised for delivering one pixmap, GEGL is
optimised for editing. I have some idea of how to optimise for viewing
by doing a simplified form of prescaling, compared to GEGL (and
hopefully outperform them both for that particular use case). Otherwise
it would be possible (but probably not always optimal) to use GEGL as a
backend for abydos then dealing with potentially large images. I need to
investigate this. Since it's common with multi gigabyte images in many
fields of science, this is something I obviously can't ignore.

> > In the near future, I'll very likely deprecate most of  
> > > GdkPixbuf's  
> > > > API, except for the I/O operations; I'd also be happy to seal
> > > > off most of its internals, within the ABI stability promise, to
> > > > avoid leakage of internal state.  
> > >
> > > Will the loader plugin API go away, you think?  
> >
> > No API will ever go away: there are no plans for a gdk-pixbuf-3.0.
> > The deprecations would mostly apply to API that is either long since
> > been replaced by Cairo (scale/composite), or that is weirdly ad hoc,
> > like saturate_and_pixelate(). Ideally, I'd like to deprecate the
> > option to build gdk-pixbuf without depending on GIO to do MIME type
> > sniffing, as GIO has been fixed to work on Windows and macOS, and it
> > is a required dependency anyway.  
>
> That's probably fine, though the current code in the stack is dumb as
> rocks, and will try to thumbnail a JPG file with the PNG decoder if
> the suffix is incorrect. The error message is also obnoxious, so
> that'll need fixing before removing the content type sniffing.

Then I ran gdk-pixbuf in a debugger I noticed that it ignored the file
name and only used the content then determining the file type. So
is there another code path which uses the filename, or is it a compile
time option? I noticed that the MIME type detection code (which I think
is more or less a copy paste of the reference implementation) uses the
filename alone to determine the MIME type if the extension is
considered reliable enough. Just as a secondary method it will use the
supplied content for sniffing. It's also possible that it returns more
than one MIME type and in that case gdk-pixbuf will always use the
first one. This is annoying since a completely made up MIME type (like
"application/x-extension-png") can be prepended to the list sometimes,
so the real one gets ignored. This is apparently a feature I don't
understand.

> >  The animation API is pretty much a garbage fire, so it may go on
> > the chopping block as well. Of course, deprecated API will keep
> > working as well—or as badly—as it works today, so people can still
> > use it. Moving pixbuf loaders to separate processes, and wrap them
> > in sandboxes, would be a fairly good thing to do; it need to be
> > decided at run time, though, because there are many users of
> > GdkPixbuf that already run in a sandbox, which prevents creating
> > smaller sandboxes inside it.  
>
> That'd be best left to an API that can do that natively, with the API
> being thought out ahead of time.
>
> In the short term, my wishlist/TODO list would be:
> - somebody writes a GIF loader that is readable, and passes the test
> suite.

I've written loader for GIF that simply wraps abydos. In lines of
code it's about a quarter the size of the current loader, even including
the GIF plugin for abydos. It might even be slightly smaller with the
whole of abydos included in the equation. On the downside it probably
doesn't pass the test suite since I haven't tried it. But I will, and
hopefully publish the whole thing in a couple of days.

> - we disable every loader by default except the ones that the core
> desktop needs
> - image viewers that rely on gdk-pixbuf ship their additional loaders
> in the app's Flatpak[1].

I don't care much for Flatpak in particular. But generalised and
rephrased as, leave it to the distributors to decide, I agree that
this is absolutely the best approach.

> On a sidenote, I also use gdk-pixbuf as a pixel-perfect drawing API
> for thumbnailers:
> https://gitlab.gnome.org/Archive/gnome-nds-thumbnailer/blob/master/gnome-nds-thumbnailer.c#L72
>
> [1]: I recently did this to add webp comics support to the evince
> Flatpak:
> https://gitlab.gnome.org/GNOME/evince/merge_requests/50

Nice!