A proposal for binary theming

[Ian Wells <I Wells tarragon-et co uk>]

I've been giving some thought to the problem of themes over the last
few days, and I've had an inspiration (which, you'll be pleased to
know, is backed up with some code).  So here's my cunning plan.

The discussion on themes in the last month or two suggests that
there's two sorts of theming.  Firstly there's the kind that
Enlightenment uses, where you theme an application basically by
supplying a load of pixmaps for it to use in various situations It's
theming basically because of the versatility of the approach.  The
other sort of theming, which I've called 'binary' theming, is where
you would override the default widget in a library (say, the GTK+
button widget) with a different one using different code (an
enlightenment-style one, for instance).  Linux's replacement Athena
widget libraries do binary theming: you change the shared lib and you
get a new widget set.

So, you have

         Application code

         Binary theme

         Theming of selected binary code
	   (pixmaps, colours, etc.)

There's another reason you might want to do binary theming other than
simply changing the appearance.  For more complex widgets like file
selection dialogs you might want to completely change the way they
work - say, you just give a drag-and-drop area and a cancel button.
You should be allowed to do this providing the result is compatible
(i.e. it allows you to select files) and you shouldn't have to change
the whole library to do it.  This way you can change the way big
chunks of your GUI work to your own taste.  It's also quite nice if
you want to test a changed widget without recompiling everything,
because done right you should be able to change part of a library in
the same way that you can override library symbol with symbols in an
object during linking.

Finally, wouldn't it be nice if you could get a program to decide for
itself what theme it wanted and to sort itself out, rather than to
force a theme on all applications?  My opinion is that you should have
some degree of granularity, so that you can change the themes of all
programs or of a single program.  For instance, the Linux shlib
approach to themes means you must link the application in a specific
way rather than against the Athena library's normal location to use a
theme other than the system-wide default.

The problem with this approach is that a program can't decide on the
shared libraries it's linked against by the dynamic linker.  If you
want to choose a library to load, you have to use dlopen(), which in
turn means you must use dlsym() to access any of the symbols within
the library, and any typechecking you'd normally get when using a
library in C is lost because dlsym() returns everything as (void *)
which you have to hand-cast to something better (which you'll
inevitably make mistakes with unless you have some automatic program
handy).  Also, note the large amount of code bodging you'd have to do
with something linked against Gtk (or with the Gtk library, if you
theme at a slightly lower level) to get it to use dlsym() on most
function calls.

Anyway, what I came up with is a way of only partially linking against 
a shared library.  A 'wrapper' program which I've written in Perl
generates stub functions for the program to link against.  These stub
functions appear to the application to be the same as the functions
actually in the library, but just look up a function pointer in a
table and jump into that instead.

The pointer table is set up and linked with a theme library to make a
shared object file.  The stubs code and pointer table code are
generated at the same time by the same program to make sure things are 
consistent.

This looks like:

         Application code

	 Stub module which looks like the binary theme API
         ------------------------ dlopen(), dlsym()
         Library wrapper which stub module uses
         Binary theme

         Theming of selected binary code
	   (pixmaps, colours, etc.)


The idea is that you can resolve the external symbols in your program
so that it'll link normally (requiring no source change if it's a Gtk
program, for instance), but you can use any shared library you choose,
selected at run time and even changeable half-way through execution.
The difference between this and normal run-time dynamic linking it
that that's done when ld is run and is final, whereas this form can
mean that even if no library is loaded the program can be executing,
and that any library which is compatible can be loaded on the fly.

(Example)
For instance, I might want to run a math library-based application
with this system.

I create my stub files:

./wrap-lib.pl /usr/lib/libm.a math_dynlib

... which wraps all public functions in the math library and creates a 
variable math_dynlib to receive the offset table from the as-yet
uncreated shared library to use.  It generates stub.S and wrapper.c.

I link my application:

gcc -o app app.c stub.S

- which doesn't have the math lib linked in but nevertheless links
fine.  app.c here needs to be aware that it's got to select and load a 
math library at some point, but otherwise it can be ignorant of the
whole issue.

The application has lines at its beginning to select the correct
version of the wrapped version of the dynamic lib, load it and put the 
pointer to the offset table in the right place.

The dynamic lib is made by:

(this command isn't the right one, it's just intended to give you the idea)
gcc -o dynlib.so wrapper.c /usr/lib/libm.a

 - which puts an offset table into the dynamic lib; or

gcc -o dynlib.so wrapper.c overrides.c /usr/lib/libm.a

which will override functions in libm.a on a module-by-module basis with
the different implementations provided in overrides.c.

Running app now:

- loads 'dynlib.so' (or another shared library linked with wrapper.c)

- gets the offset table pointer from 'dynlib.so'

- puts a pointer to it in the 'math_dynlib' variable so that the stubs 
can use it

- does maths using the stub functions which look up the real library
using the offset table pointer.

With the right code in the app, I can even unload and reload the
dynamic library as I choose.  The dynamic library's file path can even 
be loaded from the application's config. file.

There's target system dependency in my implementation of this, to keep
it simple.  In an ideal world I'd just parse all the C headers for a
library and export the functions I found there; the types I'd find
would tell me how to generate the stubs in C.  Here, I've written a
bit of i386 assembler which calculatesd the pointer to the real
routine and jumps to its start, avoiding any disturbance to the stack
or the registers so that all the arguments arrive in the called
function in their intended locations.  It's faster than using C, but
more importantly it can be done independent of the type of the
function really being called, meaning that I could put a system
together quickly and without writing a C header parser.  I actually
wrote this code on non-Linux i386 system, and it worked there, so it's
still got some limited portability.

The next step with binary theming is to find the level in Gtk+ at
which to divide it up.  You can't simply wrap the Gtk+ library code
completely in this, because one of the limnitations of this method is
that it can't export variables, only functions.  Another point is that 
if you want to be able to change the binary theme of a program during
execution, you need to

- clean up the transient data of the old theme associated with each
  Gtk object

- remove the old theme library

- load the new theme library

- get the new theme library to sort out the transient data it needs
  for each Gtk object.

But then this is probably ay in the future.

Now, any questions? ;-)  I've got some code which does the wrapping,
I'm not up to the job of deciding where to break Gtk into basic
functions and theme library.  And I might be missing something
blindingly obvious which means that you can just do this with the
dynamic linker...  Basically, I'd like feedback before I make the code 
public.

Ian.