Re: Speeding up thumbnail generation (like multi threaded). Thoughts please.

[David Zeuthen <david fubar dk>]

On Fri, 2009-08-28 at 23:15 +0200, Mark wrote:
> I haven't done io profiling but i did calculate the disc usage for
> those 1927 files. and every benchmark was WAY below what my hdd could
> handle (Spinpoint F1 1TB hdd and it can handle roughly 100MB/sec).

Uhm, wait, that's only true for sequential reading. For random IO (which
is true for thumbnailing) on rotational media, the throughput is much
much lower (probably on the order 1MB/s) due to seek times. It's much
better on SSD but not everyone has a SSD.

You also want to remember to drop all caches before profiling this - the
relevant file here is /proc/sys/vm/drop_caches - see proc(5) for more
details.

It might be useful to sort the queue of files waiting to be thumbnailed
according to the block position of the inode (see the readahead sources
for how to do that). But since a file is not necessarily laid out in
order on the disk (ext4 helps a bit here though) [1], there might be a
lot of seeking involved anyway. So it might not help at all.

(And IIRC, Nautilus already reorders the thumbnail list to give priority
to files currently visible in the file manager. It would probably look
weird if this happened out of order - so the visibility sort would win
compared to the block position sort.)

     David

[1] : And this is most definitely not true for files downloaded via
Bittorrent - which is a very normal use case for video files (which we
thumbnail)

> 
> As for the CPU. i did "profiling" there. i just opened up gnome's
> system monitor tool and just watch it during the benchmark. That gave
> the result of 1 core running at 100% and 3 other cores idle thus there
> is a cpu bottleneck with thumbnail generating.
> 
> About your suggestion for loading in a thread and thumbnailing in
> another thread.. don't you get in big problems if the pc is
> thumbnailing faster then it's loading? Or is that a impossible
> scenario?
> 
> Mark.
> _______________________________________________
> gtk-devel-list mailing list
> gtk-devel-list gnome org
> http://mail.gnome.org/mailman/listinfo/gtk-devel-list