Re: Memory statistics

[Behdad Esfahbod <behdad cs toronto edu>]

On Wed, 26 Apr 2006, Lubos Lunak wrote:

>  I don't remember any, but that doesn't mean anything. The only "problem" is
> that you can't pre-fork that much data. You can't initialize X connection,
> because forking will cause serious trouble with it, the same with any other
> file descriptor, temporary file, shared memory segment, whatever (and I don't
> know about you but initialization of a KDE application is a rather extensive
> process, so this would be difficult to check).

For GNOME the least (and probably about the most) we can do is to
register the widgets and other types with the type system.


>  There's also the issue of whether it's worth it for you. As soon as the
> forked app writes to any shared memory, you get COW immediately and the
> effect is lost. So sharing .got is fine as it's not fragmented and there
> should be no writes afterwards, but e.g. sharing heap is much worse. We also
> gain a lot from sharing vtables and moc tables, but that's only they also are
> in one block of memory and won't be written to afterwards (that's the
> data.rel.ro section, I asked one gcc developer to make gcc group things
> in .data by type). You however shouldn't have much of this either.

I'm not sure if we need it either.  On an FC5 system with xorg 7,
looking at the maps of a very simple Gtk+ Hello World:

>>> x = gtop.proc_map(20711)

there's about 900kb total private_dirty:

>>> sum (m.private_dirty for m in x)
933888L


ok which only 200kb belongs to libraries (the rest is heap,
stack, anon maps):

>>> sum (m.private_dirty for m in x if m.filename.startswith('/'))
204800L


About 50kb is just .data and GOT tables of X libraries:

>>> sum (m.private_dirty for m in x if
m.filename.startswith('/usr/lib/libX')) 53248L



[behdad home rtl]$ pmap 20711 | grep libX
00389000    996K r-x--  /usr/lib/libX11.so.6.2.0
00482000     16K rwx--  /usr/lib/libX11.so.6.2.0
00adb000     20K r-x--  /usr/lib/libXdmcp.so.6.0.0
00ae0000      4K rwx--  /usr/lib/libXdmcp.so.6.0.0
00b77000      8K r-x--  /usr/lib/libXau.so.6.0.0
00b79000      4K rwx--  /usr/lib/libXau.so.6.0.0
00bbd000     60K r-x--  /usr/lib/libXext.so.6.4.0
00bcc000      4K rwx--  /usr/lib/libXext.so.6.4.0
00ce7000     32K r-x--  /usr/lib/libXrender.so.1.3.0
00cef000      4K rwx--  /usr/lib/libXrender.so.1.3.0
00cf2000     16K r-x--  /usr/lib/libXfixes.so.3.0.0
00cf6000      4K rwx--  /usr/lib/libXfixes.so.3.0.0
00d10000     28K r-x--  /usr/lib/libXi.so.6.0.0
00d17000      4K rwx--  /usr/lib/libXi.so.6.0.0
00d1a000      8K r-x--  /usr/lib/libXinerama.so.1.0.0
00d1c000      4K rwx--  /usr/lib/libXinerama.so.1.0.0
00d2a000     36K r-x--  /usr/lib/libXcursor.so.1.0.2
00d33000      4K rwx--  /usr/lib/libXcursor.so.1.0.2
00d36000     12K r-x--  /usr/lib/libXrandr.so.2.0.0
00d39000      4K rwx--  /usr/lib/libXrandr.so.2.0.0


So we have 9 tiny X extension libraries, each about 24kb in size,
that are taking 4kb each, for a total of 36kb per process.
Without trying, but looking into them, I think linking them all
together will save at least 20kb of the 200kb...


The other 4K-data libraries are:

[behdad home rtl]$ pmap 20711 | grep rw | grep / | grep 4K | grep -v libX
0011b000      4K rwx--  /lib/libnss_files-2.4.so
0012c000      4K rwx--  /usr/lib/gtk-2.0/2.4.0/engines/libclearlooks.so
00169000      4K rwx--  /usr/lib/libpangoft2-1.0.so.0.1200.1
001b4000      4K rwx--  /usr/lib/libpangocairo-1.0.so.0.1200.1
001ce000      4K rwx--  /usr/lib/libgdk_pixbuf-2.0.so.0.800.17
00581000      4K rwx--  /usr/lib/pango/1.5.0/modules/pango-arabic-fc.so
0079d000      4K rwx--  /home/behdad/.local/lib/libgobject-2.0.so.0.1100.0
00846000      4K rwx--  /usr/lib/pango/1.5.0/modules/pango-basic-fc.so
00862000      4K rwx--  /lib/ld-2.4.so
00994000      4K rwx--  /lib/libc-2.4.so
009be000      4K rwx--  /lib/libm-2.4.so
009c4000      4K rwx--  /lib/libdl-2.4.so
009d9000      4K rwx--  /usr/lib/libz.so.1.2.3
00aca000      4K rwx--  /home/behdad/.local/lib/libglib-2.0.so.0.1100.0
00b62000      4K rwx--  /home/behdad/.local/lib/libgmodule-2.0.so.0.1100.0
00ca5000      4K rwx--  /usr/lib/libpng12.so.0.1.2.8
00e31000      4K rwx--  /home/behdad/.local/lib/libcairo.so.2.4.0


Of these, the pango modules can be compiled in the library for
example by distros.  At least for the basic module or some other
popular ones.  gmodule and gthread look offending when you look
at their text size:

[behdad home rtl]$ pmap 20711 | grep local/lib/libg
00764000    228K r-x--  /home/behdad/.local/lib/libgobject-2.0.so.0.1100.0
0079d000      4K rwx--  /home/behdad/.local/lib/libgobject-2.0.so.0.1100.0
00a38000    584K r-x--  /home/behdad/.local/lib/libglib-2.0.so.0.1100.0
00aca000      4K rwx--  /home/behdad/.local/lib/libglib-2.0.so.0.1100.0
00b60000      8K r-x--  /home/behdad/.local/lib/libgmodule-2.0.so.0.1100.0
00b62000      4K rwx--  /home/behdad/.local/lib/libgmodule-2.0.so.0.1100.0

So we have the 228kb gobject and 584kb glib libraries each have a
4k data section, and the same for the 8kb gmodule (the same is
true for gthread, though this example doesn't link to it.)  So,
why not link gmodule and gthread into libglib?


Of course example was just dragging gtk+ in.  There are other
GNOME libraries like gconf and gnomevfs that most of the
processes link to, so, I'm not saying that a launcher is not a
good idea, but without that, we can probably save some 50kb per
process by getting the tiny libs merged into larger ones and
tweaks like this.  On a typical desktop that will be like 1MB of
Federico's 14MB.

Still, in-proc applets should save a lot more.  I have eight
applet procs running, each taking 1mb of private-dirty.
Federico, can you point us to where that technology is hiding?


--behdad
http://behdad.org/

"Commandment Three says Do Not Kill, Amendment Two says Blood Will Spill"
	-- Dan Bern, "New American Language"