Hello all, Colin Walters [2011-03-25 12:18 +0100]: > == Desktop File Cache == > There are multiple ways to approach this, but I think by far the > simplest is just an mmap'able file containing all .desktop files found > in the desktop paths. Don't try to implement all the merge logic, > etc. I'm glad that this gets discussed again. Vincent Untz and I were discussing this some two years ago, as we found that reading desktop files was a serious chunk of CPU churn during session startup. Just for the record I would like to explain the approach that we have had in Ubuntu since December 09. It might not be too useful any more, but can perhaps at least give some pointers. Vincent didn't like it too much back then, and rightly said that it should be implemented using GVariant, but we never got around to this as the current system actually works quite well (and it also dropped out of focus). On a Dell Mini 10 (with an Abacus, erm, Atom processor) this saved a little less than one second startup time, and gnome-panel's CPU usage was cut to less than half. (It helped other applications which read the menus as well). This was the pre-GVariant age, and there was no appropriate serialization API available which was both fast and also not a huge pain to use. I played aorund with various file formats, and actually found that with gnome-menus a simple GKeyFile actually worked best, since in the end we want to read all the relevant desktop files anyway, and reading them linearly out of the file cache into the internal DesktopEntry tree representation was fine. The time savings came from: * Being able to ignore NoDisplay/Hide/OnlyShowIn desktop files, which eliminated about half of the .desktop files. * Not having to do gettext lookups for translations (we use gettext at runtime, so that we can update translations post-release without having to rebuild app packages, see http://bugzilla.gnome.org/show_bug.cgi?id=569829). But even with inline translations you'd save the lookups and reading a lot of extra data, so I recommend to consider per-locale caches. * Not having to readdir() /usr/share/applications/ and stat() all the files there. * On rotary disks, reading one file is much faster than reading ~ 150 files which are splattered throughout the disk. This is not that relevant with the various *readahead solutions, of course. So we build /usr/share/applications/<dirname>.<locale>.cache, which look very simple [1]. The gnome-menus patch [2] looks for these and uses that instead of parsing all the *.desktop files if present. > The next question is how does the cache get rebuilt? In the discussions back in the day we actually considered using a daemon, but we quickly discarded the idea, as this would impose quite a lot of runtime overhead, and we don't actually want to see yet another daemon in the session (which would probably eat up a lot of the CPU savings of the cache by having to inotify and stat everywhere). Instead we found: * /usr/share/applications/ has the vast majority of desktop files; /usr/local, ~/.local, etc. typically only have very few, and trying to cache them isn't worth it. * This is also the only directory which is exclusively in the "packages" domain, which we can control very well. Other dirs can be modified at any time by the user or admins. So we went for the pragmatic solution and only do caches for /usr/share/applications. (The patch above works with any desktop directory, but we only build cache files for that one). We have a script [3] to build the cache, which gets run by the package manager whenever a package adds/changes/removes/modifies files in /usr/share/applications/. (In dpkg terms this is called a "trigger"). So we only have some extra overhead at package install/upgrade time (which is already slow enough, so that the extra second doesn't matter), but no extra overhead at runtime. This concept is very hard to generalize as an upstream solution, of course. The gnome-menus patch [2] and the script [3] aren't distro specific at all, but the concept of package triggers simply doesn't exist with upstream tarballs, and thus there will always be some distro specific integration here. Also, I'm well aware of the corner case that this would simply ignore desktop files that the admin puts into /usr/share/applications/. We currently treat this case as "don't do that then", that's what /usr/local/ is for, but it's of course a small design wart of this. Thanks, Martin [1] http://people.canonical.com/~pitti/tmp/desktop.de_DE.utf8.cache [2] http://bazaar.launchpad.net/~ubuntu-desktop/gnome-menus/ubuntu/view/head:/debian/patches/22_desktop-cache.patch [3] http://bazaar.launchpad.net/~ubuntu-desktop/gnome-menus/ubuntu/view/head:/debian/update-gnome-menus-cache -- Martin Pitt | http://www.piware.de Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org)
Attachment:
signature.asc
Description: Digital signature