Re: GnomeDesktopThumbnail API replacement

[Bastien Nocera <hadess hadess net>]

On Tue, 2018-01-30 at 14:45 +0000, Debarshi Ray wrote:
> On Tue, Jan 30, 2018 at 02:06:01PM +0100, Bastien Nocera wrote:
> > On Tue, 2018-01-30 at 11:06 +0000, Debarshi Ray wrote:
> > > Also, it's not primarily about the location of the cache but the
> > > thumbnailer. To me, the real value in using the generic
> > > thumbnailers
> > > is not having to worry about all kinds of weird MIME type
> > > problems,
> > > sketchy files, and the various problems that arise from those
> > > things.
> >
> > What files are you interested in thumbnailing that your application
> > doesn't support?
>
> It's not about thumbnailing MIME types that the application doesn't
> support, no.  Sorry, I could have been clearer.
>
> It's about dealing with sandboxing the thumbnailer, dealing with
> CVEs,
> crashes and broken output arising out of weird files, etc..  Some of
> these are potentially fixable at the codec level, which is why I
> mentioned that standardizing around a smaller set of codec
> implementations will also be a win, even if the thumbnailing layer on
> top is different.
>
> (An example of a weird file would be a panorama generated by a
> Samsung
> Galaxy: https://forums.adobe.com/thread/2131432)

OK. This would however just protect you until the point you open the
image in question, unless you consider the thumbnailer's response to be
authoritative. Much like the existing sandboxing of thumbnailers won't
protect eog or evince itself, just the associated thumbnailers.

But we could make some internal changes to that API to allow you to
pass associated metadata, maybe, so you could do something like:
- tell the thumbnail API to only consider this private thumbnailer you
have, and no other
- find a way to pass 2 files, or maybe one file and a serialised
string/metadata to the thumbnailer which would do its job.

There's currently no way to do the 2nd part, but I'll try to see
whether we can make the API extensible at least, so writing something
like that is possible.

> > > One could imagine asking the generic thumbnailer to thumbnail the
> > > original file, and then have the application run the edits on top
> > > of
> > > it. However, that won't work if the image was cropped because
> > > you'd
> > > have to crop the original thumbnail and then proceed, which will
> > > affect the resolution of the final thumbnail. If I am not
> > > mistaken, I
> > > coded Photos' thumbnailer to load the image at a lower resolution
> > > if
> > > it wasn't cropped, else it loads at full scale and downscales
> > > later.
> >
> > At this point, you could very well apply the edits and thumbnail
> > internally, as the app would know which one of the photos have
> > edits
> > attached to them.
>
> Umm... I didn't understand whether you meant that as an alternative
> or
> as something to complement the existing setup.
>
> I'd like to do what you mentioned above to complement the existing
> setup.  Say you started with an empty cache or you added a new online
> account or whatever, and now you have a large number of items queued
> up for thumbnailing.  Meanwhile, if you open an unthumbnailed item at
> full resolution, then it should be possible to jump the queue and
> create a thumbnail from the hi-res.  It's just nice to do that for so
> many reasons.  Also, if you have a provider that doesn't offer
> server-generated thumbnails, then this can serve as a workaround.
>
> This is regardless of which thumbnailing API is being used.  For
> example, you can do this for Documents too, which uses the generic
> stuff; and for viewers/players and Files.  Although, it would be
> hindered by every Flatpak having its own thumbnail cache.  Do/will
> they have a shared cache?  Or not?

In Flatpak, no, it's expected that each app has its own cache.

> (One thing that I didn't mention so far is that Photos also has
> online
> thumbnailers.  They grab server-side generated thumbnails for online
> content which sometimes requires credentials for the GOA account.
> Currently these are in-process and are a glorified g_file_copy over
> HTTP.)

The thumbnailing processes are sandboxed with no network access right
now. I don't think that's something we want to change, but it just
makes it a 2-step process.

<snip>
> > > > > Last but not the least, I find it important to version the
> > > > > cache
> > > > > to tide over bugs in the thumbnailer.
> > > >
> > > > You don't need the whole cache to be versioned. Thumbnailers
> > > > can
> > > > add metadata to the thumbnails if they want, and doing:
> > > > gdk_pixbuf_save(pixbuf, "/path/to/thumbnail", "png", &error,
> > > >                 "tEXt::Thumb::Software::Version", "1.0", NULL);
> > > > is do-able from within the thumbnailer, and that extra metadata
> > > > would be kept.
> > >
> > > Yes, that's right. It will be good if the generic thumbnailers
> > > would
> > > version their output like that. Maybe it should be added to the
> > > standard?
> >
> > I'll add a "FORCE" flag to the API. As you'd control both the
> > thumbnailer and how it's called, you could do thumbnail the file,
> > and
> > if the version in the GdkPixbuf is too old, re-run the thumbnailing
> > with the "FORCE" flag to force the thumbnail to be re-created.
>
> I am confused.  Are you talking about a "FORCE" flag in the generic
> thumbnail API or the custom one-off thing in Photos?

In the thumbnail API. That's what we're discussing here, right? :)

I've already extended the API quite a bit[1], next I'll try to see
where the new code should live.

Cheers

[1]: This is the latest attempt, just the sync version:
PROPERTIES:
- whether to monitor .thumbnailer directories
- char ** directories to monitor
- timeout in seconds
- whether to cap memory usage
- GnomeDesktopThumbnailSize size
- number of threads in the thread pool

typedef enum {
    GNOME_DESKTOP_THUMBNAIL_FLAGS_NONE = 0,
    GNOME_DESKTOP_THUMBNAIL_FLAGS_SAVE,  // Whether to save the created (successful or failed) thumbnail to 
the cache
    GNOME_DESKTOP_THUMBNAIL_FLAGS_FORCE  // Whether to ignore existing thumbnails and try creating a new one
} GnomeDesktopThumbnailFlag;

#define GNOME_DESKTOP_THUMBNAIL_FILE_ATTRIBUTES                 \
        G_FILE_ATTRIBUTE_THUMBNAIL_PATH ","                     \
        G_FILE_ATTRIBUTE_THUMBNAILING_FAILED ","                \
        G_FILE_ATTRIBUTE_THUMBNAIL_IS_VALID ","                 \
        G_FILE_ATTRIBUTE_PREVIEW_ICON ","                       \
        G_FILE_ATTRIBUTE_TIME_MODIFIED ","                      \
        G_FILE_ATTRIBUTE_STANDARD_FAST_CONTENT_TYPE

GdkPixbuf *
gnome_desktop_thumbnail_generate_thumbnail_sync (GnomeDesktopThumbnailFactory  *factory,
                                                 GFile                         *file,       // instead of a 
URI?
                                                 GFileInfo                     *info,       // NULL means 
we'll look it up if necessary, otherwise must include GNOME_DESKTOP_THUMBNAIL_FILE_ATTRIBUTES, 
G_FILE_ATTRIBUTE_STANDARD_CONTENT_TYPE will however be used rather than 
G_FILE_ATTRIBUTE_STANDARD_FAST_CONTENT_TYPE if present
                                                 GnomeDesktopThumbnailFlags     flag,
                                                 char                         **ret_thumbnail_path,
                                                 GCancellable                  *cancellable,
                                                 GError                       **error);