Re: SoC idea: desktop file cache

[Alan Cox <alan lxorguk ukuu org uk>]

> There are multiple ways to approach this, but I think by far the
> simplest is just an mmap'able file containing all .desktop files found
> in the desktop paths.

This depends. First comment on the shell is "man pthreads", but if your
compositor is huge and synchronous it'll suck rocks when you are paging
anyway so you already lost the battle.

Most of the cost for reading a lot of small files is metadata reads,
particularly on laptops. In fact it's latency on the disk head movements
and rotation that does much of the damage plus (pre AHCI SATA) the single
command limit.

On a flash based drive the rules change somewhat.

In both cases a single file cures most of the latency not mmap. In fact
read() is often faster unless lots of apps will be sharing a cache. In
the read case the OS gets a single unambiguous indication of what you
want. In the mmap case it has to guess and you can end up with a lot of
fault latency - which can be even worse as it can sometimes clog up all
the threads of an app, and which is more SMP load.

mmap MAP_SHARED also makes dealing with external updating really
interesting because if the file shrinks you need to handle signal
exceptions from touching pages that no longer exist.

Thus if your operational model is for a cache kept by a single app not
all of them then having the cache file is a good idea on real disks (and
still a win on SSD) and the cache is a private copy I'd use read not mmap.

If you do need to mmap it shared then also tell the OS whether it should
preload it on the mmap (MAP_POPULATE) and whether it will be read
sequentially or not (madvise)

Alan