Re: On the cost of libraries

[Drazen Kacar <dave arsdigita com>]

Maciej Stachowiak wrote:
> On 01Sep2001 11:52PM (+0200), Drazen Kacar wrote:

> > I would strongly recommend against libraries having data which is not
> > accessed by functional interfaces. If there is such data, the library
> > must be loaded even if it will never will be used and run-time linker
> > must process data relocations.
> 
> oaf / bonobo-activation has an instance of this mistake (it exports an
> array that has the popt options rather than a function to get it). Is
> it worth breaking the API freeze on bonobo-activation to fix it?

Depends on the circumstances, ie. who is supposed to use those symbols, so
you'll have to make a decision for yourself. But if it's only an API
freeze and not binary compatibility issue, I'd break the freeze.

I'll give you a simple example for illustration.

I've just compiled XEmacs --with-gtk. XEmacs can be used as a GUI
application, in which case it would have to load some GUI library, but not
terminal handling library. It can be used as a terminal application in
which case it needs terminal handling library, but no GUI. Or it can be
used in batch mode, in which case it doesn't need any of them.

So I could mark terminal handling library and GUI library as lazy loaded
and, in theory, run-time linker would load them only if needed. In
practice, it doesn't work that way.

On Solaris, "dump -rv xemacs" shows this:

.rela.bss:
Offset      Symndx                Type              Addend

0x1ed7a0    __huge_val            R_SPARC_COPY        0
0x1eeb08    __iob                 R_SPARC_COPY        0
0x1f3764    GTK_TYPE_GDK_CURSOR_TYPE R_SPARC_COPY        0
0x1eec64    GTK_TYPE_CTREE_NODE   R_SPARC_COPY        0
0x1ee9b8    GTK_TYPE_GDK_EVENT_TYPE R_SPARC_COPY        0
0x1edeb0    GTK_TYPE_GDK_EVENT    R_SPARC_COPY        0
0x1edba4    GTK_TYPE_GDK_WINDOW   R_SPARC_COPY        0
0x1f38dc    gdk_root_window       R_SPARC_COPY        0
0x1f3c50    GTK_TYPE_GDK_COLOR    R_SPARC_COPY        0
0x2147a8    GTK_TYPE_STYLE        R_SPARC_COPY        0
0x1ee500    tzname                R_SPARC_COPY        0
0x1f377c    GTK_TYPE_GDK_FONT     R_SPARC_COPY        0
0x1ed978    __ctype               R_SPARC_COPY        0
0x1f3798    errno                 R_SPARC_COPY        0
0x1eca04    gdk_display           R_SPARC_COPY        0

Symbols of type R_SPARC_COPY are the ones which require copy
relocations. That means that the library which satisfies the dependency
cannot be lazy loaded. Why?

Libraries are usually compiled with PIC flag. The effect of that, besides
slower code, is that run-time linker can relocate all references to the
symbols the library contains. So any other library won't have problems
accessing them. But the main executable is usually not compiled with the
PIC flag and the generated code will try to access the symbol by using
some absolute address, which wouldn't work, because the library can be
loaded at any address.

In this case a workaround is performed by the linker. For every data
symbol exported by the library and used by the main executable, ld(1)
creates a symbol in BSS section of the main executable and the code in the
executable references that address. In run-time, before the application
gains control, run-time linker copies the data from the library to the BSS
section. And then it needs to go to the library itself and change all
references from the library data to BSS section, because all parts of the
process have to find the same symbol at the same address.

This must be done before the application gains control because the
run-time linker can't know when the application is going to access the
data. It's different from the function symbol lookup, because in that case
application code actually calls run-time linker, so it can do whatever is
needed at the time it's called.

In the above example symbol gdk_display is somewhat interesting. I suppose
this is what GDK_DISPLAY() macro evaluates to. I also suppose that the
reason for having this macro was a possibility of having multi-display
feature in the future, in which case the macro would evaluate to function
returning the proper data. That's a nice intention, but the current
implementation means that run-time linker always has to load GDK and
sometimes waste some time on performing relocations.

> > And all private symbols in the libraries should be marked as such.
> 
> 1) Will symbols with static linkage automatically be private? (I sure
> hope so).

Yes.

> 2) How do I make symbols with extern linkage private to the library?

Consult your linker documentation. Or all of them, if you're working on
the portable software, because linking is not a subject to any standard. Yet.

Solaris and GNU ld have -M option which takes a configuration file where
you can specify this. Even the syntax is the same. libtool has some
provisions to do this, but it lacks a way to specify some other nice
features[1] that mapfiles provide, so I wouldn't use libtool for that, if
I could choose.

[1] I could be wrong, though. The feature I'm most interested in is
    internal versioning.

-- 
 .-.   .-.    Errors have been made. Others will be blamed.
(_  \ /  _)
     |
     |        dave arsdigita com