Re: Speeding up thumbnail generation (like multi threaded). Thoughts please.

[Mark <markg85 gmail com>]

On Tue, Sep 29, 2009 at 6:00 PM, Mark <markg85 gmail com> wrote:
> On Mon, Aug 31, 2009 at 4:06 PM, Dr. Michael J. Chudobiak
> <mjc avtechpulse com> wrote:
>> On 08/31/2009 09:03 AM, jcupitt gmail com wrote:
>>>>>
>>>>> a very quick load-at-1/8th-size read where it just decompresses enough
>>>>> to be able to get the DC component of each 8x8 block. If you use
>>>>> libjpeg like this you can expect around a 100x speedup of the
>>>>> decompress step.
>>>>
>>>> The gdk-pixbuf jpeg loader does this already.
>>>
>>> That's good, but I wonder if this feature is being used? I tried this
>>> tiny program:
>>
>> I believe this is where it happens:
>>
>> http://git.gnome.org/cgit/gtk+/tree/gdk-pixbuf/io-jpeg.c#n924
>>
>>
>> - Mike
>>
>
> I've done some more testing.
> The following test results are still with the same 1927 images.
>
> ## Thumbnails generated the nautilus way from:
> gnome_desktop_thumbnail_factory_generate_thumbnail() and that function
> runs: gnome_desktop_thumbnail_scale_down_pixbuf and not
> gdk_pixbuf_scale_simple which is odd to say the least
> real    2m43.595s
> user    2m42.433s
> sys     0m0.950s
>
> Lets say this is our current 100%. for the rest of the benchmarks 100%
> is the base.
>
> ## With the function: gdk_pixbuf_scale_simple
> real    2m19.266s
> user    2m17.914s
> sys     0m1.077s
>
> Note: 117% faster then gnome_desktop_thumbnail_scale_down_pixbuf
>
>
> ## With the function: gdk_pixbuf_new_from_file_at_scale
> real    1m14.422s
> user    1m13.605s
> sys     0m0.787s
>
> Note: 220% faster then gnome_desktop_thumbnail_scale_down_pixbuf
> Note 2: 187% faster then gdk_pixbuf_scale_simple
>
>
> So, the current way: gnome_desktop_thumbnail_scale_down_pixbuf is the
> slowest possible way to go out of the available functions to scale a
> image down.
> Now i'm wondering why that function
> gnome_desktop_thumbnail_scale_down_pixbuf has been made in the forst
> place.. the comments above it state:
>
> /**
>  * gnome_thumbnail_scale_down_pixbuf:
>  * @pixbuf: a #GdkPixbuf
>  * @dest_width: the desired new width
>  * @dest_height: the desired new height
>  *
>  * Scales the pixbuf to the desired size. This function
>  * is a lot faster than gdk-pixbuf when scaling down by
>  * large amounts.
>  *
>  * Return value: a scaled pixbuf
>  *
>  * Since: 2.2
>  **/
>
> Well, the benchmarks ran above are resizing 1927 wallpaper sized
> images to a max width or height of 200 and that function clearly
> loses.
> The solution to this is extremely simple. Grab this file:
> http://git.gnome.org/cgit/gnome-desktop/tree/libgnome-desktop/gnome-desktop-thumbnail.c
> and replace the line:
>      scaled = gnome_desktop_thumbnail_scale_down_pixbuf (pixbuf,
>                                                  floor (width * scale + 0.5),
>                                                  floor (height * scale + 0.5));
> with a function that is (a lot) faster.
>
> Now i already did more benchmarking with the last function
> (gdk_pixbuf_new_from_file_at_scale) only then threaded and that adds a
> major speed boost! with me the time for thumbnailing 1927 images
> dropped to 31 seconds (552% faster then the current nautilus way) !
> but more on that in a later post.
>
> So, how do we proceed from this point?
>

Here is the benchmarking script:
http://codepad.org/hDNLjx4p

And what you need to do before you compile and run it:
There are various paths in this script like:
/home/mark/ThumbnailingBenchmarks/2000_Wallpapers-Reanimation/ so you
obviously need to change them to your image source path. Note: there
are more paths then this example!

You can run three benchmarks:
    // GLib Thumbnailing Benchmark
    //GLibThumbnailingBenchmark();

    // Glib more rapid thumbnailing benchmark
    //GLibThumbnailingBenchmarkRapid();

    //Glib threaded thumbnailing (this is the
GLibThumbnailingBenchmarkRapid benchmark only in multiple threads)
    GlibThreadedThumbnailingBenchmark();

by default the threaded benchmark is going to run but you can
obviously comment that one and run one of the other 2 benchmarks. The
difference between the GLibThumbnailingBenchmark and
GLibThumbnailingBenchmarkRapid is that the rapid one uses
gdk_pixbuf_new_from_file_at_scale where the non rapid one uses
gdk_pixbuf_new_from_file which takes significantly longer as noted in
the benchmarks.