Re: GnomeDesktopThumbnail API replacement

[Debarshi Ray <rishi is lostca se>]

On Tue, Jan 30, 2018 at 02:06:01PM +0100, Bastien Nocera wrote:
> On Tue, 2018-01-30 at 11:06 +0000, Debarshi Ray wrote:
> > Also, it's not primarily about the location of the cache but the
> > thumbnailer. To me, the real value in using the generic thumbnailers
> > is not having to worry about all kinds of weird MIME type problems,
> > sketchy files, and the various problems that arise from those things.
>
> What files are you interested in thumbnailing that your application
> doesn't support?

It's not about thumbnailing MIME types that the application doesn't
support, no.  Sorry, I could have been clearer.

It's about dealing with sandboxing the thumbnailer, dealing with CVEs,
crashes and broken output arising out of weird files, etc..  Some of
these are potentially fixable at the codec level, which is why I
mentioned that standardizing around a smaller set of codec
implementations will also be a win, even if the thumbnailing layer on
top is different.

(An example of a weird file would be a panorama generated by a Samsung
Galaxy: https://forums.adobe.com/thread/2131432)

> > One could imagine asking the generic thumbnailer to thumbnail the
> > original file, and then have the application run the edits on top of
> > it. However, that won't work if the image was cropped because you'd
> > have to crop the original thumbnail and then proceed, which will
> > affect the resolution of the final thumbnail. If I am not mistaken, I
> > coded Photos' thumbnailer to load the image at a lower resolution if
> > it wasn't cropped, else it loads at full scale and downscales later.
>
> At this point, you could very well apply the edits and thumbnail
> internally, as the app would know which one of the photos have edits
> attached to them.

Umm... I didn't understand whether you meant that as an alternative or
as something to complement the existing setup.

I'd like to do what you mentioned above to complement the existing
setup.  Say you started with an empty cache or you added a new online
account or whatever, and now you have a large number of items queued
up for thumbnailing.  Meanwhile, if you open an unthumbnailed item at
full resolution, then it should be possible to jump the queue and
create a thumbnail from the hi-res.  It's just nice to do that for so
many reasons.  Also, if you have a provider that doesn't offer
server-generated thumbnails, then this can serve as a workaround.

This is regardless of which thumbnailing API is being used.  For
example, you can do this for Documents too, which uses the generic
stuff; and for viewers/players and Files.  Although, it would be
hindered by every Flatpak having its own thumbnail cache.  Do/will
they have a shared cache?  Or not?

(One thing that I didn't mention so far is that Photos also has online
thumbnailers.  They grab server-side generated thumbnails for online
content which sometimes requires credentials for the GOA account.
Currently these are in-process and are a glorified g_file_copy over
HTTP.)

> > > > Secondly, it is important to be able to create and lookup
> > > > thumbnails
> > > > of a specific size, as opposed to enumerated constants with pre-
> > > > determined presets.
> > >
> > > This is a problem with the thumbnail spec's storage section. Jon
> > > added larger version quite some time ago, and somebody would need
> > > to go through the process to make it any bigger.
> >
> > I see, ok. By larger version, you mean one for 512x512 icons?
> >
> > I haven't had the occasion to see what a N > 2 display looks like.
> > I can imagine that upscaling 512x512 raw pixels to cover 256x256
> > logical pixels might be passable. It'd still be better than LoPPI,
> > but not better than N=2 HiPPI. I don't know.
>
> Seems to be 256x256:
> https://specifications.freedesktop.org/thumbnail-spec/thumbnail-spec-latest.html#THUMBSIZE

Oh, you meant Jon added the large 256x256 variant. Ok. :)

I had misread your original email as "Jon drafted a larger version
than 256 some time ago".  Sorry about that.

> Unfortunately, we (GTK+/GNOME) don't control the thumbnail spec. As
> mentioned, any additions would need to go through discussions on xdg-
> list.

Ok.

> > > > Last but not the least, I find it important to version the cache
> > > > to tide over bugs in the thumbnailer.
> > >
> > > You don't need the whole cache to be versioned. Thumbnailers can
> > > add metadata to the thumbnails if they want, and doing:
> > > gdk_pixbuf_save(pixbuf, "/path/to/thumbnail", "png", &error,
> > >                 "tEXt::Thumb::Software::Version", "1.0", NULL);
> > > is do-able from within the thumbnailer, and that extra metadata
> > > would be kept.
> >
> > Yes, that's right. It will be good if the generic thumbnailers would
> > version their output like that. Maybe it should be added to the
> > standard?
>
> I'll add a "FORCE" flag to the API. As you'd control both the
> thumbnailer and how it's called, you could do thumbnail the file, and
> if the version in the GdkPixbuf is too old, re-run the thumbnailing
> with the "FORCE" flag to force the thumbnail to be re-created.

I am confused.  Are you talking about a "FORCE" flag in the generic
thumbnail API or the custom one-off thing in Photos?

> > However, if all you have is one thumbnailer, then it's just
> > convenient to version the entire cache and do a "rm -rf" when the
> > version is bumped. :)
>
> And is utterly wasteful if you have a shared thumbnail cache, as your
> thumbnailer isn't the only one around.

I am confused.  The custom one-off thing in Photos is just one
thumbnailer with an app-specific cache.

Don't get me wrong.  I think that adding the Thumb::Software::Version
attribute to Photos' own thumbnails would also be nice.  They are also
missing Thumb::MTime, so one could just address those together.

> > Also, it can be useful to version the whole cache ...
> > [...]
> >
> > Anyway, I did it mostly for the convenience.
>
> See above.

I guess you might able to get away with just versioning each file, but
having some version info at various levels of granularity just seems
like a good future proof strategy to me.  It's just a matter of
specifying a number in the directory name, so nothing fancy.  You
probably won't ever need it, but if you ever do, it'll be a life
saver.

I don't know.  I have gotten really obsessed with versioning caches
and cached items while hacking on Documents and Photos.  For example,
there is no straightforward way to invalidate cached local metadata
from Tracker's DB, after you had a faulty local metadata extractor
injecting broken entries into it.  Happens all the time.  Cosimo
addressed it when he wrote the online indexers, which has been a life
saver.  Anyway, I am digressing. :)

> I don't understand the necessity of going through the thumbnailer when
> you have edits on your photos. Surely the editor that added those edits
> could save a thumbnail itself when it saves the edits to disk, no?

Yes, it can, see above, but it should also be able to deal with the
cache getting blown away.  If the cache goes away, it needs to churn
through the items and thumbnail them in the background.

At the end of the day, the thumbnailer is just a bunch of codecs to
load the file, and code to deserialize the edits and apply them on
top.  All those classes and methods are shared with the main
application.  It's just something that's split out from the main
process.

Photos' thumbnailer is just this one file:
https://gitlab.gnome.org/GNOME/gnome-photos/blob/master/src/photos-thumbnailer.c

... that implements this simple D-Bus API:
https://gitlab.gnome.org/GNOME/gnome-photos/blob/master/src/photos-thumbnailer-dbus.xml

The hardest part of it was setting up peer-to-peer D-Bus. :)