Re: Random thoughts

[Behdad Esfahbod <behdad cs toronto edu>]

On Thu, 11 Aug 2005, Roozbeh Pournader wrote:

> I'm not understanding some of what you are talking about. I'm probably
> too ignorant about memory and CPU optimizations. Would you care to
> reword or at least provide some pointers or quotes from parts of the
> fontconfig archive you are talking about?

You should have not skipped R. Love's talk :).  Here is how
fontconfig used to work:  User runs fc-cache, it will dump a
machine-readable.  The library then would parse this file on
every application start up.  The file parsed into memory, takes
around 100kb of your valuable memory.  This means 100kb per
process.  Means, if you have a GNOME desktop running, you have
some 15 copies of this 100kb in memory.  Not talking about the
CPU time it takes to create all those linked lists.

Now they have switched to an mmap()able cache format.  Lets see
why mmap is so good:


  - Never actually read the data, just mmap it.

  - Means, only the pages that you actually need are loaded
from the disk, not more.

  - Means, if there are 15 processes using the data, there's
still only one copy in memory.

  - Means, if you run out of memory, the kernel don't need to
copy the memory for mmap()ed file into swap, since it's already
there on the hard disk.

  - Means, my preload tool can prefetch it for you :).

  - Means, you avoid two times copying the data: one from kernel
buffer to stdio buffer, and once from stdio buffer to user
buffer.



So, even considering the memory overhead, we sould mmap, not
talking about how slow parsing CLDR would be.


> About fonts, the situation is different. First of all, no, they are not
> exchangable. They are not even exchangable across different Microsoft
> platforms.

I don't agree.  Considering only systems dated after 2000, fonts
are quite exchangable.


> About locale exchange, CLDR probably talks about moving things like five
> lines of configuration data around. Let's assume that in the future a
> dynamic web page may ask for the user's locale settings, and the browser
> would answer that it's CLDR 2.7's "de_DE collation=phonebook" with the
> medium date format changed to something instead of the default.

The nearest thing to this is Xkb.  As soon as we have the locale
preprocessor and compiler, it's all a matter of deciding when and
how to compile the locales.  It can be done by user's decision
(like fc-cache), on the fly, on specified periods, etc.  That's
really nothing hard.

> That's the kind of thing CLDR is thinking about, I believe. Are you
> saying that parsing that in runtime is practically impossible and should
> be avoided at all costs? Or something else?

It's not "parsing at runtime" should be avoided.  But "parsing
everytime" should be avoided.  The first time you parse, you also
dump it to mmap the next time.  Simple.



--behdad
http://behdad.org/