Re: Speeding up thumbnail generation (like multi threaded). Thoughts please.

[Alexander Larsson <alexl redhat com>]

On Wed, 2009-09-30 at 15:27 +0200, Philip Van Hoof wrote:
> On Fri, 2009-08-28 at 22:11 +0200, Mark wrote:
> 
> > 
> > Now for the results:
> > 
> > Glib
> > ----------------------
> > 1927 images thumbnailed in 2.29 minutes. That is roughly 0.07 seconds
> > per thumbnail
> > 
> > GraphicsMagick
> > ----------------------
> > 1927 images thumbnailed in 3.08 minutes. That is roughly 0.09 seconds
> > per thumbnail
> > 
> > FreeImage
> > ----------------------
> > 1927 images thumbnailed in 5.45 minutes. That is roughly 0.17 seconds
> > per thumbnail
> 
> You want to override GdkPixbuf's support for Jpeg and thumbnail all JPEG
> images using EPeg. It'll beat GdkPixbuf by 5 times or something.
> 
> That's because afaik does EPeg use libjpeg in a way that it skips rows
> and columns, and that way performs better on both I/O (having to read
> less of the original), memory (same reason) and CPU (having to
> decompress less, having to scale less or nothing).
> 
> We are already using EPeg for this on the Maemo Fremantle platform, by
> the way. Scaling truly large JPEG images goes magically fast with it.

I'm not sure EPeg is really that magic. 
For instance, this: http://www.freshports.org/graphics/epeg says:

  It makes use of libjpeg features of being able to load an image by only
  decoding the DCT coefficients needed to reconstruct an image of the size
  desired. This gives a massive speedup. If you do not try and access the pixels
  in a format other than YUV (or GRAY8 if the source is grascale) then it also
  avoids colorspace conversions as well.

The first part (DCT stuff) we already do in the jpeg loader (its the
"jpeg loader trick" talked about above). The second we could easily do
ourself in the jpeg loader for scaled image loads by scaling in
YUV/GRAY8 space before converting.

I'd love if someone would like to work on this in gdk-pixbuf.

Another thumbnailing performance trick is to use thumbnails in EXIF
data, which many cameras add, if availible. This is slightly more
dubious to do in gdk-pixbuf as the thumbnail is not 100% reliable (i.e.
it may be out of date due to file modifications or whatnot). So,
something like this really has to be done in the gnome-desktop
thumbnailer code.