Re: Memory statistics

[Lubos Lunak <l lunak suse cz>]

On Saturday 22 April 2006 02:50, Federico Mena Quintero wrote:
> On Sat, 2006-04-22 at 00:16 +0200, David Turner wrote:
> > This has a huge impact on C++ libraries, since all their VTables contain
> > constant pointers (try running
> > your script on a KDE system !).

 *sigh*

>
> ... or on OpenOffice.  Michael "I want to be like him when I grow up -
> oh, wait, he's younger than me" Meeks has been kicking some ass in that
> front.

 Not really. If you're talking about -Bdirect, that one doesn't affect memory 
usage at all. It only reduces the number of .so's where a symbol is looked 
for, but the number of relocations processed is exactly the same.

> > It'd be nice if the dynamic linker was able to automatically share the
> > updated pointers between
> > processes which map the same libraries to the same virtual addresses,
> > but I don't think this has been
> > implemented yet, since it would require a new ELF section, as well as
> > modifications to the
> > compilers.
>
> Wouldn't something like this need to happen:
>
> 	1. program gets run; linker does the fixups
>
> 	2. linker writes "libfoo.so got put at 0x12345678, here is a
>            list of resolved relocations: <...>"
>
> 	3. another program gets run.
>
> 	4. linker looks in its archive from (2) to see if it had
> 	   already resolved libfoo.so at some address.
>
> Sounds like pain.

 As Matthias already pointed out, also sounds a lot like prelink :) , although 
not the same. The problems should be more or less the same though, and given 
that the one already existing solution with its current pace of development 
will be practically usable around 2020 [*], it's probably not wise to start 
another one without thinking twice first :(.

 [*] The fact that it's been in Gentoo since ages doesn't count. It still has 
a number of practical problems and especially when it gets to memory usage it 
can't even hold a candle to kdeinit. I should even have some numbers on this 
in my blog or somewhere (for KDE apps indeed), I could try to dig it up if 
there'd be interest. In theory (at least I believe) those problems should be 
fixable with (quite) some amount of work, this work doesn't seem to be 
currently the reality however :(.

> > Another option is to get rid of PIC code completely, using an alternate
> > dynamic linking scheme,
> > similar to the one in Windows, without the crap. If rebasing is used,
> > relocation can be reduced to
> > nearly nothing for the most common libraries. This would also create
> > smaller and faster code.
>
> How do they handle the problem of agreeing on the base addresses for
> various libraries?  We have a million libraries in Linux :)

 (Just guessing.) I think they don't. They instead handle the problem of 
coping with address conflicts by simply relocating elsewhere. The memory 
usage increase caused by this relatively rare case should be more than paid 
off by saving many dirty pages of the most common libraries that shouldn't 
cause conflicts.

-- 
Lubos Lunak
KDE developer
---------------------------------------------------------------------
SuSE CR, s.r.o.  e-mail: l lunak suse cz , l lunak kde org
Drahobejlova 27  tel: +420 2 9654 2373
190 00 Praha 9   fax: +420 2 9654 2374
Czech Republic   http://www.suse.cz/