Re: SoC idea: desktop file cache

Hello all,

Colin Walters [2011-03-25 12:18 +0100]:
> == Desktop File Cache ==
> There are multiple ways to approach this, but I think by far the
> simplest is just an mmap'able file containing all .desktop files found
> in the desktop paths.  Don't try to implement all the merge logic,
> etc.

I'm glad that this gets discussed again. Vincent Untz and I were
discussing this some two years ago, as we found that reading desktop
files was a serious chunk of CPU churn during session startup.

Just for the record I would like to explain the approach that we have
had in Ubuntu since December 09. It might not be too useful any more,
but can perhaps at least give some pointers. Vincent didn't like it
too much back then, and rightly said that it should be implemented
using GVariant, but we never got around to this as the current system
actually works quite well (and it also dropped out of focus).

On a Dell Mini 10 (with an Abacus, erm, Atom processor) this saved a
little less than one second startup time, and gnome-panel's CPU usage
was cut to less than half. (It helped other applications which read
the menus as well).

This was the pre-GVariant age, and there was no appropriate
serialization API available which was both fast and also not a huge
pain to use. I played aorund with various file formats, and actually
found that with gnome-menus a simple GKeyFile actually worked best,
since in the end we want to read all the relevant desktop files
anyway, and reading them linearly out of the file cache into the
internal DesktopEntry tree representation was fine. The time savings
came from:

 * Being able to ignore NoDisplay/Hide/OnlyShowIn desktop files, which
   eliminated about half of the .desktop files.

 * Not having to do gettext lookups for translations (we use gettext
   at runtime, so that we can update translations post-release without
   having to rebuild app packages, see But even with
   inline translations you'd save the lookups and reading a lot of
   extra data, so I recommend to consider per-locale caches.

 * Not having to readdir() /usr/share/applications/ and stat() all the
   files there.

 * On rotary disks, reading one file is much faster than reading ~ 150
   files which are splattered throughout the disk. This is not that
   relevant with the various *readahead solutions, of course.

So we build /usr/share/applications/<dirname>.<locale>.cache, which
look very simple [1]. The gnome-menus patch [2] looks for these and
uses that instead of parsing all the *.desktop files if present.

> The next question is how does the cache get rebuilt?

In the discussions back in the day we actually considered using a
daemon, but we quickly discarded the idea, as this would impose quite
a lot of runtime overhead, and we don't actually want to see yet
another daemon in the session (which would probably eat up a lot of
the CPU savings of the cache by having to inotify and stat
everywhere). Instead we found:

 * /usr/share/applications/ has the vast majority of desktop files;
   /usr/local, ~/.local, etc. typically only have very few, and
   trying to cache  them isn't worth it. 

 * This is also the only directory which is exclusively in the
   "packages" domain, which we can control very well. Other dirs can
   be modified at any time by the user or admins.
So we went for the pragmatic solution and only do caches for
/usr/share/applications. (The patch above works with any desktop
directory, but we only build cache files for that one). We have a
script [3] to build the cache, which gets run by the package manager
whenever a package adds/changes/removes/modifies files in
/usr/share/applications/. (In dpkg terms this is called a "trigger").
So we only have some extra overhead at package install/upgrade time
(which is already slow enough, so that the extra second doesn't
matter), but no extra overhead at runtime.

This concept is very hard to generalize as an upstream solution, of
course. The gnome-menus patch [2] and the script [3] aren't distro
specific at all, but the concept of package triggers simply doesn't
exist with upstream tarballs, and thus there will always be some
distro specific integration here.

Also, I'm well aware of the corner case that this would simply ignore
desktop files that the admin puts into /usr/share/applications/. We
currently treat this case as "don't do that then", that's what
/usr/local/ is for, but it's of course a small design wart of this.



Martin Pitt                        |
Ubuntu Developer (  | Debian Developer  (

Attachment: signature.asc
Description: Digital signature

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]