Introspection

[Matthias Clasen <mclasen redhat com>]

I'd like to get the Introspection effort going again, with the 
goal of getting something ready for GLib 2.8. To spark the discussion, 
here is a quick braindump of on this topic. Note that I haven't
looked in depth at the prototype work done by Rob Melby yet, so
a lot of this may already be done or at least begun (I hope 
Rob and Mathrick will jump into the discussion to clear this up).



The Goal should be to allow people to write fully introspectable 
objects (for the D-BUS bindings, for scriptable apps interfaces, etc.)
and to allow language bindings to get rid of all the all the .defs
and header scanning that is being done currently. There should be
a way to programmatically invoke the described functions. We need to
provide metadata for classes, interfaces, functions (both methods of 
classes and freestanding functions)

We should probably not cover all irregularities of C apis (varargs come
to mind).

What information is needed ?

Functions
* name, maybe need a separate symbol-name
* link methods to classes
* link constructors to classes
* link getters and setters to properties
* type information for parameters and return value, including
  - is a pointer really an array ? if so, is it NULL-terminated,
    or is there another parameter holding its length ? 
  - for GLib container types, we need to allow parameter types
  - can the parameter be NULL, or can NULL be returned ?
* direction of parameters: in, out or both ?
* memory management for parameters and return value, including
  full vs. shallow ownership for container types (as in: free
  the returned list, but not the content)
* which parameters should be optional, and what are the default 
  values for optional parameters

Classes/Interfaces
* methods 
  - same information as for functions
* virtual functions
  - same information as for functions
  - offset in class struct
* signals
  - same information as for functions
  - offset of default handler in class struct
* properties
  - information already provided by gobject

Boxed types
* fields 
  - type
  - struct offset


Implementation ideas
* The metadata should be stored as a binary blob in the library.
  one idea for finding the metadata is to use a symbol name  
  derived from the module name, e.g. the metadata for GTK+ would
  be accessible via a symbol like _GTK_METADATA. Using a fixed
  symbol name would be problematic when using dlsym() to find
  the data. 

  The binary format should be extensible. There are different
  ways to achieve this, e.g. tagging all fields (allows to
  skip unknown tags), or using a fixed layout and store size 
  information for each entry (allows to skip unknown stuff
  at the end of the entry).

  Strings should probably be kept in a string pool.

* The type information needed for the parameter and field 
  descriptions goes beyond what gobject provides, since we
  need to be able to say, e.g. GList of GtkWidget*. We probably
  don't need a full recursive type system, though.

* GType type ids are registered at runtime (except for a few
  fixed ones), therefore type names must be stored in the on-disk
  metadata. 

  In order to get around the problem that types can
  not be looked up until they are registered, we need to store
  the name of the get_type function in addition to the type name.
  
  To make cross-library references work, we must also store the
  module name for types from other modules. Eg. the GTK+ metadata
  blob should store the reference to the PangoLayout type as
  "Pango", "PangoLayout", "pango_layout_get_type".

* It should be possible to convert between the binary metadata
  format and an xml format. This is very useful for debugging.

* It may not be necessary to store the struct offsets explicitly
  in the metadata blow. If the members of classes/boxed types are
  described in order, its is possible to write platform-specific
  code to calculcate the offsets. In any case, there should 
  probably be a way to find out the struct offsets, so that 
  language bindings don't have to implement the struct layout
  calculations themselves.

* Metadata for classes/interfaces/boxed types should probably be
  associated with the GType so that it can be looked up in that way,
  but we also need a way to enumerate the complete metadata for
  a module, e.g. to get access to metadata for plain functions.
 

Open questions
* Do cross-module references work as outlined above ? 

* Should the property metadata which is already available in 
  gobject be duplicated ?

* Should we explode the binary metadata into structs or just 
  navigate directly on the binary data ?

* Should documentation be included in the metadata ?
  
* Should type information for parametrized container types
  be provided on the gobject level, or should we do a simple
  ad-hoc description for such types ?


Matthias