Re: [Fwd: libpng shared memory patch]

[Lubos Lunak <l lunak suse cz>]

On Saturday 22 April 2006 01:32, Federico Mena Quintero wrote:
> Andrea Bedini has started kicking ass!  This constifies a bunch of
> things in libpng.
>
>   Federico
> Zapouzdřená zpráva
>   libpng shared memory patch
> Od:
> "Andrea Bedini" <andrea bedini gmail com>
>   Komu:
> federico gnu org
>   Datum:
> Dnes 00:58:56
> Hi,
> in response to your
> http://primates.ximian.com/~federico/news-2006-04.html#footprint-private-di
>rty
>
> thank you for writing a such a simple and understandable explanation
> of sharing mechanism. I piked up a small library like libpng a I tried
> to analyze its exposed structs, looking for some const to place. I got
> one third reduction of memory footprint, but I think we can reduce it
> further. Here are the details and a patch.

 Boy, does this bring memories.

> libpng version 1.2.8rel-5 from dapper
>
> before:
>
> $ objdump -x /usr/lib/libpng.so  | grep data
>   11 .rodata       00002a00  00021a30  00021a30  00021a30  2**3
>  18 .data         00000348  00034cd0  00034cd0  00024cd0  2**2
>   20 .sdata        000000b0  0003502c  0003502c  0002502c  2**2
>
> 0x348+0xb0=0x3F8=1016 bytes
>
> $ python memstats.py | grep libpng
> private: /usr/lib/libpng12.so.0.1.2.8: 168 KB (spread among 84 mappings)

 This in other words means that every process has two dirty pages because of 
libpng.

> shared: /usr/lib/libpng12.so.0.1.2.8: 16 KB (spread among 84 mappings)
>
> (with 107 processes around)
>
> after:
>
> $ objdump -x libpng12.so| grep .data | head
>   11 .rodata       00002d40  00021a30  00021a30  00021a30  2**3
>  18 .data         00000008  0003589c  0003589c  0002589c  2**2
>   20 .sdata        000000b0  000358b8  000358b8  000258b8  2**2
>
> 0x340 = 832 bytes moved from .data to .rodata; maybe another 100 bytes
> can be moved.
>
> $ python memstats.py | grep libpng
> private: /usr/lib/libpng12.so.0.1.2.8: 100 KB (spread among 81 mappings)
> shared: /usr/lib/libpng12.so.0.1.2.8: 8 KB (spread among 81 mappings)
>
> (with 100 processes around)
>
> size of .data + .sdata has been reduced from about 1Kb to about 0.2Kb
> this means 80%, why the final result if just 30% ? maybe a lot of
> processes use the 20% that remains in .sdata ?

 Because the basics of this math are wrong. You cannot count in bytes, you 
have to count in memory pages (4K on x86). If you change dirty "bytes" in a 
library from 8000 to 4100, you save nothing. If you change it from 4100 to 
4090, you save one memory page, going down from two to one. Moreover it's 
probably practically impossible to get no dirty page at all for a library 
mapping, so you cannot get below this 4K (which is the case with libpng here 
according to exmap and to the sum of non-READONLY ALLOC ELF section sizes, no 
idea why it initially takes two for you).

-- 
Lubos Lunak
KDE developer
---------------------------------------------------------------------
SuSE CR, s.r.o.  e-mail: l lunak suse cz , l lunak kde org
Drahobejlova 27  tel: +420 2 9654 2373
190 00 Praha 9   fax: +420 2 9654 2374
Czech Republic   http://www.suse.cz/