Re: On the cost of libraries



Drazen Kacar <dave arsdigita com> writes:

> Owen Taylor wrote:
> 
> > Let's first get some hard data.
> 
> What kind of data would you prefer?

Timing data. Map data tells me little more than some details about
how tables are set up in memory; it doesn't give any real indication
of what the cost is of setting up those tables is.
 
> > I find it a little hard to believe that exporting a single variable
> > causes more than a single variables worth runtime link overhead.
> 
> Just the first one, in certain circumstances. Any additional symbol
> doesn't incur overhead.

Certainly there is _some_ overhead. The more copy relocs I
have to do, the more pages of the shared library I have to touch...
 
> I can offer xemacs as an example.
> 
> "ldd xemacs" shows:
> 
>         libtiff.so.1 =>  /usr/local/lib/libtiff.so.1
>         libpng.so.2 =>   /usr/local/lib/libpng.so.2
>         libjpeg.so.1 =>  /usr/local/lib/libjpeg.so.1
>         libgdk_imlib.so.1 =>     /usr/local/lib/libgdk_imlib.so.1
>         libgtk-1.2.so.0 =>       /usr/local/lib/libgtk-1.2.so.0
>         libgdk-1.2.so.0 =>       /usr/local/lib/libgdk-1.2.so.0
>         libX11.so.4 =>   /usr/lib/libX11.so.4
>         libglib-1.2.so.0 =>      /usr/local/lib/libglib-1.2.so.0
>         libdl.so.1 =>    /usr/lib/libdl.so.1
>         libdb-3.2.so =>  /usr/local/lib/libdb-3.2.so
>         libgdbm.so.1 =>  /usr/local/lib/libgdbm.so.1
>         libcurses.so.1 =>        /usr/lib/libcurses.so.1
>         libldap.so.4 =>  /usr/lib/libldap.so.4
>         libm.so.1 =>     /usr/lib/libm.so.1
>         libsocket.so.1 =>        /usr/lib/libsocket.so.1
>         libnsl.so.1 =>   /usr/lib/libnsl.so.1
>         libgen.so.1 =>   /usr/lib/libgen.so.1
>         libc.so.1 =>     /usr/lib/libc.so.1
>         libz.so.1 =>     /usr/lib/libz.so.1
>         libXext.so.0 =>  /usr/lib/libXext.so.0
>         libgif.so.3 =>   /usr/local/lib/libgif.so.3
>         libgmodule-1.2.so.0 =>   /usr/local/lib/libgmodule-1.2.so.0
>         libgnuintl.so.1 =>       /usr/local/lib/libgnuintl.so.1
>         libresolv.so.2 =>        /usr/lib/libresolv.so.2
>         libmp.so.2 =>    /usr/lib/libmp.so.2
>         libdga.so.1 =>   /usr/openwin/lib/libdga.so.1
>         /usr/platform/SUNW,Ultra-250/lib/libc_psr.so.1
> 
> Then, on "xemacs -nw" process, mapping is as follows:
> 
> 18260:  ./xemacs -nw
>  Address   Kbytes Resident Shared Private Permissions       Mapped File
> 00010000    1784    1784    1568     216 read/exec         xemacs
> 001DC000    3488    3488    1408    2080 read/write/exec   xemacs
> 00544000     288     288       -     288 read/write/exec     [ heap ]
> FEC70000     168     112      40      72 read/exec         libcurses.so.1
> FECAA000      32      32       -      32 read/write/exec   libcurses.so.1
> FECB2000       8       8       -       8 read/write/exec   libcurses.so.1
> FECC0000      24      24      16       8 read/exec         nss_files.so.1
> FECD6000       8       8       -       8 read/write/exec   nss_files.so.1
> FECE0000      16      16       8       8 read/exec         libmp.so.2
> FECF4000       8       8       -       8 read/write/exec   libmp.so.2
> FED00000     552     432     352      80 read/exec         libnsl.so.1
> FED9A000      32      32       -      32 read/write/exec   libnsl.so.1
> FEDA2000      32      24       -      24 read/write/exec   libnsl.so.1
> FEDD0000      72      32       -      32 read/exec         libdga.so.1
> FEDF2000       8       8       -       8 read/write/exec   libdga.so.1
> FEE00000      40      32      24       8 read/exec         libsocket.so.1
> FEE1A000       8       8       -       8 read/write/exec   libsocket.so.1
> FEE20000      96      16       8       8 read/exec         libXext.so.0
> FEE48000       8       8       -       8 read/write/exec   libXext.so.0
> FEE50000      88      88      80       8 read/exec         libm.so.1
> FEE74000       8       8       -       8 read/write/exec   libm.so.1
> FEE80000     560     224      64     160 read/exec         libX11.so.4
> FEF1C000      24      24       -      24 read/write/exec   libX11.so.4
> FEF30000      24      16       -      16 read/exec         libgnuintl.so.1
> FEF44000      16      16       -      16 read/write/exec   libgnuintl.so.1
> FEF50000     184     120       -     120 read/exec         libglib-1.2.so.0
> FEF8C000      32      24       -      24 read/write/exec   libglib-1.2.so.0
> FEFA0000     248      96       -      96 read/exec         libgdk-1.2.so.0
> FEFEC000      32      32       -      32 read/write/exec   libgdk-1.2.so.0
> FF000000    1976     712       -     712 read/exec         libgtk-1.2.so.0
> FF1FC000     296     136       -     136 read/write/exec   libgtk-1.2.so.0
> FF250000       8       8       -       8 read/write/exec     [ anon ]
> FF260000       8       8       -       8 read/exec         libgmodule-1.2.so.0
> FF270000       8       8       -       8 read/write/exec   libgmodule-1.2.so.0
> FF280000     672     608     592      16 read/exec         libc.so.1
> FF338000      24      24       -      24 read/write/exec   libc.so.1
> FF33E000       8       8       -       8 read/write/exec   libc.so.1
> FF350000       8       8       -       8 read/write/exec     [ anon ]
> FF360000      16      16       8       8 read/exec         libc_psr.so.1
> FF380000       8       8       -       8 read/exec         libdl.so.1
> FF390000       8       8       8       - read/shared       dev:32,4 ino:202286
> FF3A0000       8       8       -       8 read/write/exec     [ anon ]
> FF3B0000     136     136     128       8 read/exec         ld.so.1
> FF3E2000       8       8       -       8 read/write/exec   ld.so.1
> FFBE4000      48      48       -      48 read/write/exec     [ stack ]
> 
> You will notice a certain number of libraries missing, because they just
> weren't used yet. There's GTK and all its dependencies because I didn't
> use lazy loading when linking libgtk, so all the libraries it depends on
> were pulled in.
> 
> ldd /usr/local/lib/libgtk.so
>         libgdk-1.2.so.0 =>       /usr/local/lib/libgdk-1.2.so.0
>         libgmodule-1.2.so.0 =>   /usr/local/lib/libgmodule-1.2.so.0
>         libglib-1.2.so.0 =>      /usr/local/lib/libglib-1.2.so.0
>         libX11.so.4 =>   /usr/lib/libX11.so.4
>         libm.so.1 =>     /usr/lib/libm.so.1
>         libgnuintl.so.1 =>       /usr/local/lib/libgnuintl.so.1
>         libc.so.1 =>     /usr/lib/libc.so.1
>         libXext.so.0 =>  /usr/lib/libXext.so.0
>         libdl.so.1 =>    /usr/lib/libdl.so.1
>         libsocket.so.1 =>        /usr/lib/libsocket.so.1
>         libnsl.so.1 =>   /usr/lib/libnsl.so.1
>         libdga.so.1 =>   /usr/openwin/lib/libdga.so.1
>         libmp.so.2 =>    /usr/lib/libmp.so.2
>         /usr/platform/SUNW,Ultra-250/lib/libc_psr.so.1
> 
> Hmm, I suppose the next time I'll use lazy loading for GTK as well. Even
> if it has to be loaded, libX11 and libXext (and hopefully some others) don't.
> Although this could probably go into bug report.
> 
> Anyway, all those Guppi libraries are perfectly fine for this OS, as long
> as they don't cause copy relocations. They just wouldn't be loaded until
> needed.

ELF on Solaris, in my understanding, is very similar to ELF on
Linux. 

Because of the way symbol lookups in ELF works, even if there are no
copy relocations pointing to a library, the first time the linker
needs to look up a symbol that is in a library after those libraries
in the link order, it's going to load up those libraries.

That is, if I have   gcc -o a.out main.c -lfoo -lbar

then if main.c references bar_a(), then there is no way the runtime
linker can avoid loading libfoo to see if bar_a() is in
libfoo. 

AFAIK, ELF doesn't record _which_ shared library a reference is in,
just that the  reference is in some external shared library. 
(Which has advantages ... under ELF, you can move symbols around
between shared libs without breaking bin compat, and you have
the LD_PRELOAD capability.) 

What we are discussing here is not the difference between no
guppi libs and 10 guppi libs, but the difference between 1
big guppi library and 10 small guppi libraries. For this,
copy relocs, extra, aren't the relevant factor. The relevant
factor is basically where they are in the link order. If
they are at the end of the link order, then the 10 library
split only affects lookups within the libraries; if they
are in the middle of the link order, they affect lookups for
every symbol after the libraries.

But, as Alex showed, we aren't talking a huge deal here for
the slowdown either way.  

Also, I'll point out, your example is quite unusual. It's not
often you have an executable that links to GTK+ but doesn't
use in most cases. So, the fact that copy relocs hurt this
case isn't a very urgent consideration for me. 

In some cases, exporting variables can result in considerably
more efficient code ... consider if 'stdin' involved a 
function call for every use, or the typical implementation
of ctype().... GNU libc exports about 60 variables, and
we have a number of important exports in GLib as well.

00051f08 D g_ascii_table
00051f60 D g_thread_functions_for_glib_use
00051f58 D g_threads_got_initialized
00059520 D g_utf8_skip

Being the important ones. (Just checked in a fix so that
g_ascii_table and g_utf8_skip are 4 byte copy relocs,
rather than 512 and 256 byte copy relocs.)   

And finally, remember a copy reloc doesn't get added to the
a program unless the program actually accesses the variable.
Most programs don't have a copy reloc for gdk_display,
because most programs don't use the GDK_DISPLAY() macro.

Regards,
                                        Owen 

_______________________________________________
gnome-hackers mailing list
gnome-hackers gnome org
http://mail.gnome.org/mailman/listinfo/gnome-hackers




[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]