Re: Beagle CPU usage (was Proposed module: tracker)

[Federico Mena Quintero <federico ximian com>]

El mi�28-03-2007 a las 01:11 +0100, Bastien Nocera escribi�> I'll note that even if it doesn't do anything, modifying the
> thumbnailers to not mmap the files they're trying to thumbnail goes
> against your original point of the thumbnailers being third-party
> applications we had no control over.

Well, thumbnailer programs are in a funny situation.  Up to now I think
all of them have been written by careful GNOME hackers, but the
thumbnailers often depend on "external" libraries which don't get our
usual amount of love.

The problem we had at Novell was rather embarrassing:

1. We shipped libgsf (from Gnumeric et al) with its usual thumbnailer
for MS Office files (yeah, I wrote that crap) :)

2. The thumbnailer does this:

	- use libgsf to look in the OLE stream for the thumbnail

	- extract the thumbnail, which comes as a Windows Metafile

	- call convert(1) to convert the WMF into a PNG

	- write the PNG to the appropriate location

3. The code in libgsf and its thumbnailer is pretty robust, but the
underlying code in convert(1) is not.  It uses libwmf and some other
crazy library to render vectors into bitmaps, and the combination is
what most cultures have defined as "brittle".

4. In a customer site, which has tons and tons of MS Office files, the
thumbnailer was seen to consume all memory when thumbnailing certain
files from PowerPoint.  This was on a server for thin clients, so *all*
the client sessions got swapped to hell; disaster ensued.

5. I looked into fixing convert(1) and its libraries, but there was just
too much code for an urgent fix.  Hence the idea of a safety valve in
gsf-office-thumbnailer.

What is funny is that it was *not obvious at all* why the thumbnailer
was getting run.  The user didn't even click on the culprit file with
Nautilus.  They opened the file from within OpenOffice.  This caused the
recent-files list to get updated, which in turn caused gnome-main-menu
to look at the new list of recent files, which caused it to try to
generate a thumbnail (which you wouldn't be able to see, since
gnome-main-menu hadn't even been invoked).

My point is that thumbnailers get run at random times, they have the
scary job of handling bizarre files, they depend on code which we don't
control, and it is a good idea to have a safety valve in place.  Since
*we* define the requirements for thumbnailers ("put this GConf key here,
install your program there"), we can certainly mention in the
documentation that they need to be able to run within a constrained
environment.

Another option is to add a GConf key for the limits.  Right now we
have /desktop/gnome/thumbnailers/<mimetype>/{command,enable}.  We could
add "cpu_limit" and "mem_limit" under /thumbnailers.  This is probably
overkill, but it would let the sysadmin tweak and lock down the safety
valve herself.

  Federico