Re: Memory statistics

[Lubos Lunak <l lunak suse cz>]

On Tuesday 18 April 2006 05:36, Federico Mena Quintero wrote:
> Hi,
>
> I started writing a small script that should eventually let us answer
> questions like these:
>
> - How much memory are we using in the writable mappings for libraries?
> Is it worthwhile to hunt down non-const arrays and such?
>
> - How much memory are we using for the heaps of all processes?
>
> - How much memory
>
> The script is attached.  You run it like
>
> 	python ./memstats.py
>
> Right now it looks at all of your user's pids, parses /proc/<pid>/smaps
> for them, and adds up the private_dirty and shared_dirty values for each
> mapping.  It prints the sorted results in two sections: one for
> private_dirty, one for shared_dirty.
>
> For example, if /usr/lib/libfoo.so has 8 KB of private_dirty and 20
> processes use that library, you'll see a line like
>
> 	/usr/lib/libfoo.so: 160
>
> which is (8 * 20).
>
> I'm going to bed right now, but if someone wants to make the script
> produce more useful stats, go ahead and go crazy :)

 Hmm ... how exactly is that supposed to be different from exmap[*] :) ? The 
Files view can tell you that too - e.g. my /usr/lib/libpng.so.3.1.2.8 is used 
by 55 processes (Number of procs), occupying total of 340K (Effective 
Resident or Effective Mapped) out of which 84K is dirty (Writable). It can 
even display the mappings down to the level of ELF sections (besides all the 
other useful things exmap can do).

[*] 
http://mail.gnome.org/archives/performance-list/2005-November/msg00073.html

 Also, from your blog:

> This alone amounts to almost 16 MB. My basic desktop was running (i.e. 
nautilus, panel, etc.) along with Evolution, Epiphany, and Gaim. By adding 
const here and there we can save at least 16 MB from the footprint of such a 
basic desktop.

 Sorry, I'm afraid that's just wishful thinking. There are 2080 mappings in 
total in the table and assuming each mapping is equivalent to one process 
using the library, with one 4K memory page per mapping, that's 8M, half of 
your at-least-to-be-saved-16M. It's very unlikely any library will be with no 
dirty pages at all (you'll very likely always have at least one relocation, 
one variable or something), so you can't get below this. The only way reduce 
this 8M seems to be to reduce the numbers you get from "ldd gnomeapp | wc 
-l", which seems to be against current practices - X.org has separate .so for 
every silly thing, GNOME seems to use a lot of .so's too, and even for KDE 
apps this number isn't much lower although we try to keep the number of .so's 
low.

-- 
Lubos Lunak
KDE developer
---------------------------------------------------------------------
SuSE CR, s.r.o.  e-mail: l lunak suse cz , l lunak kde org
Drahobejlova 27  tel: +420 2 9654 2373
190 00 Praha 9   fax: +420 2 9654 2374
Czech Republic   http://www.suse.cz/