Thumbnailing service project; opinions, suggestions?

[Mark <markg85 gmail com>]

Hi,

on school i study Computer Science. That exists of a dozen "Major"
semesters and a few "Minor" semesters. Each lasts 10 weeks and that
for 3 years.
So this project, described in depth below, will have to be done within
10 school weeks of which the first 8 are the "creation" weeks. After
that it gets rated by my teachers and i either pass or fail it.
Now on school i want to learn the following coding methods in a Minor
part. A minor is a part where i "can" compose my own semester and i
did it with the following "demands" as in methods i want to learn in
that semester:

•       Plugin architecture
•       Asynchronous
•       Multi-threaded
•       Design patterns
•       Data structures (hash, queue, stack etc...)

And the methods that will be in it but already "known" to me

•       Daemon
•       IPC (D-Bus)

Now i've read the wish on this list to make a thumbnailing service for
a generic central way to generate thumbnails.
All of the above methods can be used in a thumbnailing service. So, i
made a proposal on school to make just that, a thumbnailing service
with a plugin architecture.

Is this been done before?
-------------------------
Well, not done before but a group of people is doing it as we speak:
http://live.gnome.org/ThumbnailerSpec ... i mailed them roughly this
mail a week ago to see if we could work together on this but i sadly
had to conclude that their project is in the final stages before release
1 so there would only be documenting and bug fixing. That is also part
of my semester but not the only part. I want to learn more then just
bug hunting and documentation stuff and for that there is no room in
their project thus i ended up concluding that i can't join them for the
simple fact that my school would probably not allow that as a project.

So, in roughly 13 weeks time (3 weeks in the current semester then the
10 weeks for this project) the Gnome community will have the ability to
choose between 2 thumbnailing services. That is not a maybe but a
certainty. Why? Well, i made this proposal on school, it got approved so
now i simply have to make it in order to pass the semester.

Will i be compatible with their dbus thumbnailing spec? Perhaps..
perhaps not. I do need to design the project before i start making the
actual code (major part of the project) but i do not know if i will
stick to the dbus rules they made. I can see some problems coming there
if you have huge folders to thumbnail that flood dbus.. O well, time
tell the answer to this.

Thumbnailing service
-------------------------
The idea in short is:
The daemon holds the plugin framework and knows how to handle certain
things (to be determined) defined by class templates and interfaces.
A plugin can then for example say something like this (in readable
language instead of code):
"Hey, i'm the FFMPEG plugin and i can handle the following extensions
<<long list of extensions>>"
Plugins should be placed in a folder after which a daemon update
(through dbus?) should be executed to load and activate the plugins.

So, what you get with this is a thumbnailing service that has plugins
(the pure service only holds the plugin architecture and lists from
extensions to plugins provided by the plugins).
That also means there will be a pixbuf plugin to read the default glib
images. (more on this one later after optimizing)

Another thing which i see as needed but is not part of this school
project is making a allowed/denied list in the daemon that contains
the extensions allowed to thumbnail besides the extensions available to
make thumbnails from. The idea is that you tell the thumbnailing daemon
through dbus or by reading gconf keys which extensions are allowed
(or not allowed).

Then for the dbus part. The idea is to let a application (for example
nautilus) request the thumbnails from their full scale file paths.
then the daemon should return the new file path to the thumbnails. That
is one way but could potential be a issue with folders that contain
thousands of files.

Another way to do this is: let nautilus send the folder that needs to
be thumbnailed. then the daemon should read the folder, check if a
thumbnail already exists and make it if not. The daemon should then
(as long as it's busy) send updates (let’s say every 0.1 second) to
nautilus with the files that have been thumbnailed.

What probably might be a better idea is to let nautilus look in the
folder and see which files need to be thumbnailed instead of handling
that task to the daemon. Then nautilus can send the list of files that
need to be thumbnailed to the daemon.

Yet another possible way where i have no clue if it's possible is
making a file list hash (or map or vector or. anything) in nautilus that
contains the files that needs to be thumbnailed. Nautilus should filter
out the ones that are already thumbnailed. Then the pointer of that hash
(let’s call it a hash for now) should be send to the thumbnailing daemon
where the thumbnails are being made and returned in another hash
(also provided by nautilus). I have no idea if this is for one even
possible and if it's wise to just pointers from one app to another.
If this gets to complex for the scope of this school project then the
simple solution (aka the first method) will be used.

The intention (and requirement) is to make this daemon asynchronous so
it won't block if another application suddenly wants to request
thumbnails.

Optimizing
-------------------------
Since the project didn't seem complex enough on paper (can you believe
that!) i was forced to add one more thing. Luckily i already wanted to
do something with optimizing in Gnome/glib so i added it to the paper.

The idea with the optimizing is making the thumbnail generation as
fast as possible! If you've been reading the gtk devel list lately you
must have noticed that i already did quite a lot to make thumbnail
generation as fast as possible. The latest benchmark so far:
http://img36.imageshack.us/i/scalingperformance.png/
which shows a significant speed improvement when using threaded
thumbnailing. This mean that the intended optimizing part is already
partly done but could be improved more by using threaded thumbnailing
with gnome_thumbnail_scale_down_pixbuff in combination with
gdk_pixbuff_new_from_file_at_scale.

About the image; with the 70 seconds bar my cpu cores (4) ran at 40%
each and i had "50%" iowait in iostat. The 21 seconds is when the
images are cached then all cores ran at 100% resulting at 21 seconds.
The benchmarks are all done with the bilinear flag set where possible
for the best quality thumbnails.

When that's working in the daemon it might be possible to speed up the
rendering of the thumbnails in nautilus (the gtk part of it) by batch
rendering the available thumbnails if that's not done already. Let’s say
a nautilus redraw every 0.2 seconds till all thumbnails are visible.

There are more optimiz ways like using OpenCL or Cairo! more about them
just below.

GdkPixbuf? Cairo? OpenCL?
-------------------------
First GdkPixbuf. I have been told that the current GdkPixbuf "sucks" and
should probably be avoided when attempting to make thumbnails as fast
as possible. One of the reasons to avoid it where Cairo. If i wanted to
have hardware accelerated downscaling i first need to make a Cairo
object and from there i need to make a Xlib object. Then i can save it
with cairo but only in .png and i personally want to be able to pick
JPEG2000 and JPEG as well. Now i have been told that only the steps of
converting a GdkPixbuf to a Xlib object could take so long that there is
hardly speed to gain there. So for that i would have to avoid GdkPixbuf
and take a more direct aproach. Note that this does fall outside the
scope of this school project but it would just be nice to have.

As for OpenCl. I am havily in favor of that method but my hardware
configuration is sadly the most horrible one to choose. Intel CPU
and nvidia GPU. there are no stable releases for the GPU yet on linux
and for the CPU there isn't a driver so i have no OpenCL options.
However if this becomes possible during my project i will certainly
attempt to implement it.

Repository?
-------------------------
I would love to have a repository for this on the Gnome servers. Would
that be possible? it would be my first ever GIT repo. If this is not
going to be allowed then it's either going to be a github, gitorious or
google code repository. The code will be publicly available with the
GPL (v2 or v3) license.

Blog or Log?
-------------------------
I also would like to make daily or weekly posts of the progress. I can
do that on my own blog but would it then be allowed that my blog is
readable on planet gtk or planet gnome (or both)?

Gnome subdomain?
-------------------------
If the Repository can't be with gnome i will probably just use my site
whatever those other 3 repo providers offer. But i would like to have
a gnome sub domain for example: thumbnailing-service.gnome.org or some
other name (i still have to make up a name for this project).

Conclusion
-------------------------
As said there was the wish to make this. I made this proposal on
school and it got accepted. This means i __have__ to make it now.
That's a risk but in the worst case scenario i just gain knowledge with
the methods above. In reality i hope this to be useful and perhaps even
viable for inclusion in Gnome. It might result in a rewrite to be good
enough for gnome ^_^. My intention certainly is to make it usable in
gnome whether that is as just a app of mine or a app distributed with
every new gnome release.. the latter option would be the best one.

The approved school document
-------------------------
I sadly can't give it. It's written in dutch. If you are dutch or want
to translate it to english you can get it. It's 12 pages of which 3 are
scheduling pages which can be trashed. The text in this mail describes
it more in depth then the approved document since that needed to be
"general".

Feedback
-------------------------
I hope some people could post their opinions about this, where can
more optimizing things be done that fall in this scope, did i miss
features for the thumbnialing service? etc.. etc.. Your feedback would
be greatly appreciated.

Thanx a lot for reading this long mail,
Mark