Re: Thumbnailing service project; opinions, suggestions?

[Mark <markg85 gmail com>]

Hi,

Let me say this again. (said it but apparently not clear)
I am currently in a school semester which takes roughly 3 weeks till
that one is done.
Then the next semester starts (and each semester takes 10 weeks) which
is the thumbnailing service semester.
So i only have 10 weeks not 13.
of those 10 weeks the first 8 are actual making, designing, and
documenting. the last 2 weeks are assessment and bug fixing when
needed.

So, i have __8__ weeks not 13.

For the complexity. What i described above is WAY to much for 8 weeks
(or 10 or 13) but those are for some part wishes (like OpenCL
inclusion, Cairo, pixbuf pointer passing).
The bare essentials are to make a (simple) service with some
functionality, make a plugin architecture, make plugins, optimize it
and "done"... (that includes c++ methods like inheritance, overloading
and making use of data structures like queue and stack but that's easy
since i already did that in my performance benchmark as listed in my
post (image).

And to clear up a misunderstanding about the idea. The idea is set
now, it has been approved and now it HAS to be made. i can not switch
ideas anymore. what i described in my first post if what is going to
be submitted on school at the ens of the next semester. I can focus
more on performance and plugins (like a video plugin) and leave the
daemon for what it is (nothing) till far in the project since i'm in
command of my own project.. just as long as the promised pieces get
submitted in "quality".

For Jannis and Philip,
Nice tumbler talk but could you two make suggestions for my project? ^_^

On Mon, Oct 19, 2009 at 5:20 PM, Rob Taylor <rob taylor codethink co uk> wrote:
> Hi Mark,
>
> We already have a thumbnailing service that is only just now starting to
> be used. A complete reimplementation of soemthing existing will not be a
> very useful thing. I also suggest that 13weeks for your entire plan is
> very over optimistic. I would suggest a better option would be to do a
> library with optimised thumbnailing routines. This would also be very
> useful for existng projects and probably result in you code being used
> in a number of situations. To add complexity you could also branch out
> into making thumbnail clips for videos. If you write up such a plan
> completely, i think you'd find it's more than enough work for a 13 week
> project.
>
> Thanks,
> Rob
>
> Mark wrote:
>> Hi,
>>
>> on school i study Computer Science. That exists of a dozen "Major"
>> semesters and a few "Minor" semesters. Each lasts 10 weeks and that
>> for 3 years.
>> So this project, described in depth below, will have to be done within
>> 10 school weeks of which the first 8 are the "creation" weeks. After
>> that it gets rated by my teachers and i either pass or fail it.
>> Now on school i want to learn the following coding methods in a Minor
>> part. A minor is a part where i "can" compose my own semester and i
>> did it with the following "demands" as in methods i want to learn in
>> that semester:
>>
>> •       Plugin architecture
>> •       Asynchronous
>> •       Multi-threaded
>> •       Design patterns
>> •       Data structures (hash, queue, stack etc...)
>>
>> And the methods that will be in it but already "known" to me
>>
>> •       Daemon
>> •       IPC (D-Bus)
>>
>> Now i've read the wish on this list to make a thumbnailing service for
>> a generic central way to generate thumbnails.
>> All of the above methods can be used in a thumbnailing service. So, i
>> made a proposal on school to make just that, a thumbnailing service
>> with a plugin architecture.
>>
>> Is this been done before?
>> -------------------------
>> Well, not done before but a group of people is doing it as we speak:
>> http://live.gnome.org/ThumbnailerSpec ... i mailed them roughly this
>> mail a week ago to see if we could work together on this but i sadly
>> had to conclude that their project is in the final stages before release
>> 1 so there would only be documenting and bug fixing. That is also part
>> of my semester but not the only part. I want to learn more then just
>> bug hunting and documentation stuff and for that there is no room in
>> their project thus i ended up concluding that i can't join them for the
>> simple fact that my school would probably not allow that as a project.
>>
>> So, in roughly 13 weeks time (3 weeks in the current semester then the
>> 10 weeks for this project) the Gnome community will have the ability to
>> choose between 2 thumbnailing services. That is not a maybe but a
>> certainty. Why? Well, i made this proposal on school, it got approved so
>> now i simply have to make it in order to pass the semester.
>>
>> Will i be compatible with their dbus thumbnailing spec? Perhaps..
>> perhaps not. I do need to design the project before i start making the
>> actual code (major part of the project) but i do not know if i will
>> stick to the dbus rules they made. I can see some problems coming there
>> if you have huge folders to thumbnail that flood dbus.. O well, time
>> tell the answer to this.
>>
>> Thumbnailing service
>> -------------------------
>> The idea in short is:
>> The daemon holds the plugin framework and knows how to handle certain
>> things (to be determined) defined by class templates and interfaces.
>> A plugin can then for example say something like this (in readable
>> language instead of code):
>> "Hey, i'm the FFMPEG plugin and i can handle the following extensions
>> <<long list of extensions>>"
>> Plugins should be placed in a folder after which a daemon update
>> (through dbus?) should be executed to load and activate the plugins.
>>
>> So, what you get with this is a thumbnailing service that has plugins
>> (the pure service only holds the plugin architecture and lists from
>> extensions to plugins provided by the plugins).
>> That also means there will be a pixbuf plugin to read the default glib
>> images. (more on this one later after optimizing)
>>
>> Another thing which i see as needed but is not part of this school
>> project is making a allowed/denied list in the daemon that contains
>> the extensions allowed to thumbnail besides the extensions available to
>> make thumbnails from. The idea is that you tell the thumbnailing daemon
>> through dbus or by reading gconf keys which extensions are allowed
>> (or not allowed).
>>
>> Then for the dbus part. The idea is to let a application (for example
>> nautilus) request the thumbnails from their full scale file paths.
>> then the daemon should return the new file path to the thumbnails. That
>> is one way but could potential be a issue with folders that contain
>> thousands of files.
>>
>> Another way to do this is: let nautilus send the folder that needs to
>> be thumbnailed. then the daemon should read the folder, check if a
>> thumbnail already exists and make it if not. The daemon should then
>> (as long as it's busy) send updates (let’s say every 0.1 second) to
>> nautilus with the files that have been thumbnailed.
>>
>> What probably might be a better idea is to let nautilus look in the
>> folder and see which files need to be thumbnailed instead of handling
>> that task to the daemon. Then nautilus can send the list of files that
>> need to be thumbnailed to the daemon.
>>
>> Yet another possible way where i have no clue if it's possible is
>> making a file list hash (or map or vector or. anything) in nautilus that
>> contains the files that needs to be thumbnailed. Nautilus should filter
>> out the ones that are already thumbnailed. Then the pointer of that hash
>> (let’s call it a hash for now) should be send to the thumbnailing daemon
>> where the thumbnails are being made and returned in another hash
>> (also provided by nautilus). I have no idea if this is for one even
>> possible and if it's wise to just pointers from one app to another.
>> If this gets to complex for the scope of this school project then the
>> simple solution (aka the first method) will be used.
>>
>> The intention (and requirement) is to make this daemon asynchronous so
>> it won't block if another application suddenly wants to request
>> thumbnails.
>>
>> Optimizing
>> -------------------------
>> Since the project didn't seem complex enough on paper (can you believe
>> that!) i was forced to add one more thing. Luckily i already wanted to
>> do something with optimizing in Gnome/glib so i added it to the paper.
>>
>> The idea with the optimizing is making the thumbnail generation as
>> fast as possible! If you've been reading the gtk devel list lately you
>> must have noticed that i already did quite a lot to make thumbnail
>> generation as fast as possible. The latest benchmark so far:
>> http://img36.imageshack.us/i/scalingperformance.png/
>> which shows a significant speed improvement when using threaded
>> thumbnailing. This mean that the intended optimizing part is already
>> partly done but could be improved more by using threaded thumbnailing
>> with gnome_thumbnail_scale_down_pixbuff in combination with
>> gdk_pixbuff_new_from_file_at_scale.
>>
>> About the image; with the 70 seconds bar my cpu cores (4) ran at 40%
>> each and i had "50%" iowait in iostat. The 21 seconds is when the
>> images are cached then all cores ran at 100% resulting at 21 seconds.
>> The benchmarks are all done with the bilinear flag set where possible
>> for the best quality thumbnails.
>>
>> When that's working in the daemon it might be possible to speed up the
>> rendering of the thumbnails in nautilus (the gtk part of it) by batch
>> rendering the available thumbnails if that's not done already. Let’s say
>> a nautilus redraw every 0.2 seconds till all thumbnails are visible.
>>
>> There are more optimiz ways like using OpenCL or Cairo! more about them
>> just below.
>>
>> GdkPixbuf? Cairo? OpenCL?
>> -------------------------
>> First GdkPixbuf. I have been told that the current GdkPixbuf "sucks" and
>> should probably be avoided when attempting to make thumbnails as fast
>> as possible. One of the reasons to avoid it where Cairo. If i wanted to
>> have hardware accelerated downscaling i first need to make a Cairo
>> object and from there i need to make a Xlib object. Then i can save it
>> with cairo but only in .png and i personally want to be able to pick
>> JPEG2000 and JPEG as well. Now i have been told that only the steps of
>> converting a GdkPixbuf to a Xlib object could take so long that there is
>> hardly speed to gain there. So for that i would have to avoid GdkPixbuf
>> and take a more direct aproach. Note that this does fall outside the
>> scope of this school project but it would just be nice to have.
>>
>> As for OpenCl. I am havily in favor of that method but my hardware
>> configuration is sadly the most horrible one to choose. Intel CPU
>> and nvidia GPU. there are no stable releases for the GPU yet on linux
>> and for the CPU there isn't a driver so i have no OpenCL options.
>> However if this becomes possible during my project i will certainly
>> attempt to implement it.
>>
>> Repository?
>> -------------------------
>> I would love to have a repository for this on the Gnome servers. Would
>> that be possible? it would be my first ever GIT repo. If this is not
>> going to be allowed then it's either going to be a github, gitorious or
>> google code repository. The code will be publicly available with the
>> GPL (v2 or v3) license.
>>
>> Blog or Log?
>> -------------------------
>> I also would like to make daily or weekly posts of the progress. I can
>> do that on my own blog but would it then be allowed that my blog is
>> readable on planet gtk or planet gnome (or both)?
>>
>> Gnome subdomain?
>> -------------------------
>> If the Repository can't be with gnome i will probably just use my site
>> whatever those other 3 repo providers offer. But i would like to have
>> a gnome sub domain for example: thumbnailing-service.gnome.org or some
>> other name (i still have to make up a name for this project).
>>
>> Conclusion
>> -------------------------
>> As said there was the wish to make this. I made this proposal on
>> school and it got accepted. This means i __have__ to make it now.
>> That's a risk but in the worst case scenario i just gain knowledge with
>> the methods above. In reality i hope this to be useful and perhaps even
>> viable for inclusion in Gnome. It might result in a rewrite to be good
>> enough for gnome ^_^. My intention certainly is to make it usable in
>> gnome whether that is as just a app of mine or a app distributed with
>> every new gnome release.. the latter option would be the best one.
>>
>> The approved school document
>> -------------------------
>> I sadly can't give it. It's written in dutch. If you are dutch or want
>> to translate it to english you can get it. It's 12 pages of which 3 are
>> scheduling pages which can be trashed. The text in this mail describes
>> it more in depth then the approved document since that needed to be
>> "general".
>>
>> Feedback
>> -------------------------
>> I hope some people could post their opinions about this, where can
>> more optimizing things be done that fall in this scope, did i miss
>> features for the thumbnialing service? etc.. etc.. Your feedback would
>> be greatly appreciated.
>>
>> Thanx a lot for reading this long mail,
>> Mark
>> _______________________________________________
>> gtk-devel-list mailing list
>> gtk-devel-list gnome org
>> http://mail.gnome.org/mailman/listinfo/gtk-devel-list
>
>
> --
> Rob Taylor, Codethink Ltd. - http://codethink.co.uk
>