Re: On the cost of libraries
- From: Owen Taylor <otaylor redhat com>
- To: Drazen Kacar <dave arsdigita com>
- Cc: Alex Larsson <alexl redhat com>, Maciej Stachowiak <mjs noisehavoc org>, gnome-hackers gnome org
- Subject: Re: On the cost of libraries
- Date: 02 Sep 2001 13:38:31 -0400
Drazen Kacar <dave arsdigita com> writes:
> Owen Taylor wrote:
>
> > Let's first get some hard data.
>
> What kind of data would you prefer?
Timing data. Map data tells me little more than some details about
how tables are set up in memory; it doesn't give any real indication
of what the cost is of setting up those tables is.
> > I find it a little hard to believe that exporting a single variable
> > causes more than a single variables worth runtime link overhead.
>
> Just the first one, in certain circumstances. Any additional symbol
> doesn't incur overhead.
Certainly there is _some_ overhead. The more copy relocs I
have to do, the more pages of the shared library I have to touch...
> I can offer xemacs as an example.
>
> "ldd xemacs" shows:
>
> libtiff.so.1 => /usr/local/lib/libtiff.so.1
> libpng.so.2 => /usr/local/lib/libpng.so.2
> libjpeg.so.1 => /usr/local/lib/libjpeg.so.1
> libgdk_imlib.so.1 => /usr/local/lib/libgdk_imlib.so.1
> libgtk-1.2.so.0 => /usr/local/lib/libgtk-1.2.so.0
> libgdk-1.2.so.0 => /usr/local/lib/libgdk-1.2.so.0
> libX11.so.4 => /usr/lib/libX11.so.4
> libglib-1.2.so.0 => /usr/local/lib/libglib-1.2.so.0
> libdl.so.1 => /usr/lib/libdl.so.1
> libdb-3.2.so => /usr/local/lib/libdb-3.2.so
> libgdbm.so.1 => /usr/local/lib/libgdbm.so.1
> libcurses.so.1 => /usr/lib/libcurses.so.1
> libldap.so.4 => /usr/lib/libldap.so.4
> libm.so.1 => /usr/lib/libm.so.1
> libsocket.so.1 => /usr/lib/libsocket.so.1
> libnsl.so.1 => /usr/lib/libnsl.so.1
> libgen.so.1 => /usr/lib/libgen.so.1
> libc.so.1 => /usr/lib/libc.so.1
> libz.so.1 => /usr/lib/libz.so.1
> libXext.so.0 => /usr/lib/libXext.so.0
> libgif.so.3 => /usr/local/lib/libgif.so.3
> libgmodule-1.2.so.0 => /usr/local/lib/libgmodule-1.2.so.0
> libgnuintl.so.1 => /usr/local/lib/libgnuintl.so.1
> libresolv.so.2 => /usr/lib/libresolv.so.2
> libmp.so.2 => /usr/lib/libmp.so.2
> libdga.so.1 => /usr/openwin/lib/libdga.so.1
> /usr/platform/SUNW,Ultra-250/lib/libc_psr.so.1
>
> Then, on "xemacs -nw" process, mapping is as follows:
>
> 18260: ./xemacs -nw
> Address Kbytes Resident Shared Private Permissions Mapped File
> 00010000 1784 1784 1568 216 read/exec xemacs
> 001DC000 3488 3488 1408 2080 read/write/exec xemacs
> 00544000 288 288 - 288 read/write/exec [ heap ]
> FEC70000 168 112 40 72 read/exec libcurses.so.1
> FECAA000 32 32 - 32 read/write/exec libcurses.so.1
> FECB2000 8 8 - 8 read/write/exec libcurses.so.1
> FECC0000 24 24 16 8 read/exec nss_files.so.1
> FECD6000 8 8 - 8 read/write/exec nss_files.so.1
> FECE0000 16 16 8 8 read/exec libmp.so.2
> FECF4000 8 8 - 8 read/write/exec libmp.so.2
> FED00000 552 432 352 80 read/exec libnsl.so.1
> FED9A000 32 32 - 32 read/write/exec libnsl.so.1
> FEDA2000 32 24 - 24 read/write/exec libnsl.so.1
> FEDD0000 72 32 - 32 read/exec libdga.so.1
> FEDF2000 8 8 - 8 read/write/exec libdga.so.1
> FEE00000 40 32 24 8 read/exec libsocket.so.1
> FEE1A000 8 8 - 8 read/write/exec libsocket.so.1
> FEE20000 96 16 8 8 read/exec libXext.so.0
> FEE48000 8 8 - 8 read/write/exec libXext.so.0
> FEE50000 88 88 80 8 read/exec libm.so.1
> FEE74000 8 8 - 8 read/write/exec libm.so.1
> FEE80000 560 224 64 160 read/exec libX11.so.4
> FEF1C000 24 24 - 24 read/write/exec libX11.so.4
> FEF30000 24 16 - 16 read/exec libgnuintl.so.1
> FEF44000 16 16 - 16 read/write/exec libgnuintl.so.1
> FEF50000 184 120 - 120 read/exec libglib-1.2.so.0
> FEF8C000 32 24 - 24 read/write/exec libglib-1.2.so.0
> FEFA0000 248 96 - 96 read/exec libgdk-1.2.so.0
> FEFEC000 32 32 - 32 read/write/exec libgdk-1.2.so.0
> FF000000 1976 712 - 712 read/exec libgtk-1.2.so.0
> FF1FC000 296 136 - 136 read/write/exec libgtk-1.2.so.0
> FF250000 8 8 - 8 read/write/exec [ anon ]
> FF260000 8 8 - 8 read/exec libgmodule-1.2.so.0
> FF270000 8 8 - 8 read/write/exec libgmodule-1.2.so.0
> FF280000 672 608 592 16 read/exec libc.so.1
> FF338000 24 24 - 24 read/write/exec libc.so.1
> FF33E000 8 8 - 8 read/write/exec libc.so.1
> FF350000 8 8 - 8 read/write/exec [ anon ]
> FF360000 16 16 8 8 read/exec libc_psr.so.1
> FF380000 8 8 - 8 read/exec libdl.so.1
> FF390000 8 8 8 - read/shared dev:32,4 ino:202286
> FF3A0000 8 8 - 8 read/write/exec [ anon ]
> FF3B0000 136 136 128 8 read/exec ld.so.1
> FF3E2000 8 8 - 8 read/write/exec ld.so.1
> FFBE4000 48 48 - 48 read/write/exec [ stack ]
>
> You will notice a certain number of libraries missing, because they just
> weren't used yet. There's GTK and all its dependencies because I didn't
> use lazy loading when linking libgtk, so all the libraries it depends on
> were pulled in.
>
> ldd /usr/local/lib/libgtk.so
> libgdk-1.2.so.0 => /usr/local/lib/libgdk-1.2.so.0
> libgmodule-1.2.so.0 => /usr/local/lib/libgmodule-1.2.so.0
> libglib-1.2.so.0 => /usr/local/lib/libglib-1.2.so.0
> libX11.so.4 => /usr/lib/libX11.so.4
> libm.so.1 => /usr/lib/libm.so.1
> libgnuintl.so.1 => /usr/local/lib/libgnuintl.so.1
> libc.so.1 => /usr/lib/libc.so.1
> libXext.so.0 => /usr/lib/libXext.so.0
> libdl.so.1 => /usr/lib/libdl.so.1
> libsocket.so.1 => /usr/lib/libsocket.so.1
> libnsl.so.1 => /usr/lib/libnsl.so.1
> libdga.so.1 => /usr/openwin/lib/libdga.so.1
> libmp.so.2 => /usr/lib/libmp.so.2
> /usr/platform/SUNW,Ultra-250/lib/libc_psr.so.1
>
> Hmm, I suppose the next time I'll use lazy loading for GTK as well. Even
> if it has to be loaded, libX11 and libXext (and hopefully some others) don't.
> Although this could probably go into bug report.
>
> Anyway, all those Guppi libraries are perfectly fine for this OS, as long
> as they don't cause copy relocations. They just wouldn't be loaded until
> needed.
ELF on Solaris, in my understanding, is very similar to ELF on
Linux.
Because of the way symbol lookups in ELF works, even if there are no
copy relocations pointing to a library, the first time the linker
needs to look up a symbol that is in a library after those libraries
in the link order, it's going to load up those libraries.
That is, if I have gcc -o a.out main.c -lfoo -lbar
then if main.c references bar_a(), then there is no way the runtime
linker can avoid loading libfoo to see if bar_a() is in
libfoo.
AFAIK, ELF doesn't record _which_ shared library a reference is in,
just that the reference is in some external shared library.
(Which has advantages ... under ELF, you can move symbols around
between shared libs without breaking bin compat, and you have
the LD_PRELOAD capability.)
What we are discussing here is not the difference between no
guppi libs and 10 guppi libs, but the difference between 1
big guppi library and 10 small guppi libraries. For this,
copy relocs, extra, aren't the relevant factor. The relevant
factor is basically where they are in the link order. If
they are at the end of the link order, then the 10 library
split only affects lookups within the libraries; if they
are in the middle of the link order, they affect lookups for
every symbol after the libraries.
But, as Alex showed, we aren't talking a huge deal here for
the slowdown either way.
Also, I'll point out, your example is quite unusual. It's not
often you have an executable that links to GTK+ but doesn't
use in most cases. So, the fact that copy relocs hurt this
case isn't a very urgent consideration for me.
In some cases, exporting variables can result in considerably
more efficient code ... consider if 'stdin' involved a
function call for every use, or the typical implementation
of ctype().... GNU libc exports about 60 variables, and
we have a number of important exports in GLib as well.
00051f08 D g_ascii_table
00051f60 D g_thread_functions_for_glib_use
00051f58 D g_threads_got_initialized
00059520 D g_utf8_skip
Being the important ones. (Just checked in a fix so that
g_ascii_table and g_utf8_skip are 4 byte copy relocs,
rather than 512 and 256 byte copy relocs.)
And finally, remember a copy reloc doesn't get added to the
a program unless the program actually accesses the variable.
Most programs don't have a copy reloc for gdk_display,
because most programs don't use the GDK_DISPLAY() macro.
Regards,
Owen
[
Date Prev][
Date Next] [
Thread Prev][
Thread Next]
[
Thread Index]
[
Date Index]
[
Author Index]
