Re: Introspection API

[Matthias Clasen <mclasen redhat com>]

On Fri, 2005-02-25 at 01:39 +0100, Maciej Katafiasz wrote:
> Dnia 24-02-2005, czw o godzinie 15:20 -0500, Matthias Clasen napisał:
> 
> > GObject binary metadata
> > -----------------------
> > 
> > GObject metadata is stored in a variable with the name _G_METADATA in
> > the shared 
> > object it describes. The format of the data is strongly influenced by
> > the Mozilla
> > XPCOM format. 
> 
> >From what Mike said, referencing any variable in ELF means instant ABI
> breakage, since somehow linking to variable relies on its size. Should
> change that to private ELF segment, or better yet, just say it's detail
> and up to implementation (as on Win32 for example you can't add
> arbitrary segments to PE, from what I know)
> 

Oh, this is certainly meant to be an implementation detail, not part of
the API. The library-side api to this stuff should be the 

g_irepository_register   (const guchar *metadata);
g_irepository_unregister (const guchar *metadata);

functions which were included in the repository API draft sent out
earlier. I would envision the metadata compiler to emit a C file
containing _init and _fini functions calling the above functions.

I have currently no idea how ELF sections work. How does that work
practically ? Is it portable ?

> [snip]
> 
> > Types are specified by two bytes. If the high byte is zero, the low
> > byte 
> > describes a basic type, otherwise the 16bit number is an offset
> > relative to 
> > header->types and points to a TypeBlob. 
> 
> Hmm, so we will need to always pad first 255 bytes in header->types,
> because no TypeBlob can point to anything below 256 byte, right?
> 

Yes, the first 255 bytes after the location pointed to by header->bytes
are not usable for types. I was envisioning to just let header->bytes 
point 255 bytes "too low", if you care about the little wasted space.

> [snip]
> 
> > ValueBlob (32 bytes)
> > 
> > Values commonly occur in enums and flags, but we also allow them to
> > occur
> > in interfaces or freestanding, to describe constants.
> > 
> > struct ValueBlob
> > {
> >   guint16 type;  /* 9 */
> >   guint   deprecated : 1;
> >   guint   reserved   :15;
> >   guint32 name;
> > 
> >   guint32 short_name;
> >   guint32 value;
> > }
> > 
> > short_name: 
> >           A short name for the value;
> > 
> > value:    The numerical value;
> 
> Are we we going to only ever need numerical values? What about other
> types? Or did I misinterpret something?
> 
> If not, I'd propose layout like this (less compact, but we won't be
> storing millions of consts in metadata I suppose):
> 

Well, not millions, but GTK+ has certainly a non-neglegible number of 
enumeration values.

> struct ValueBlob
> {
>   guint16 type;  /* 9 */
>   guint   deprecated : 1;
>   guint   reserved   :15;
>   guint32 name;
> 
>   guint32 short_name;
>   guint32 value_length;
>   guint16 type; /* TypeBlob */
>   guint8 value[]; /* Interpret according to type */
> }
> 

Ok, I clearly overlooked the fact that we may want to allow non-numeric
constants. To explain how this happened: ValueBlob was originally only
for enumeration/flags values, where only numerical values make sense.
Then Michael Meeks pointed out the need to describe constants, so I
thought allowing values outside of enumerations would take care of that.

Maybe we should have a separate ConstantBlob including type information.

> 
> On related note, it would be probably useful to be able to get
> non-function C symbol names, wouldn't it? So slam additional symbol_name
> in above struct. In case C symbol is there, we also want to avoid
> specifying value ourselves (length == 0), to keep metadata and actual
> value seen by C code in sync.

What would you use it for ? The symbol names are necessary to look up
the functions, and the type names are necessary to look up GTypes. But
I don't see a real use for symbol names here, unless you want to
describe exported variables, which I had not in mind at all.
Another possible reason to include C names would be if we wanted to 
be able to regenerate the headers from the metadata, but I'm not sure
that is really worthwhile or practical.

> [snip]
> 
> > struct InterfaceBlob
> > {
> >   guint16 type;  /* 4 for object, 5 for interface, 6 for enum, 7 for
> > flags, 8 for boxed */
> >   guint   deprecated   : 1; 
> >   guint   reserved     :15;
> >   guint32 name; 
> > 
> >   guint32 type_name;
> >   guint32 type_init;
> > 
> >   guint16 n_interfaces;
> >   guint16 n_values;
> >   guint16 n_fields;
> >   guint16 n_properties;
> >   guint16 n_methods;
> >   guint16 n_signals;
> >   guint16 n_virtual_functions;
> > 
> >   guint16 parent;
> >   guint16 interfaces[];
> >  
> >   ValueBlob           values[];    
> >   FieldBlob           fields[];
> >   PropertyBlob        properties[];
> >   FunctionBlob        methods[];
> >   SignalBlob          signals[];
> >   VirtualFunctionBlob virtual_functions[];
> > } 
> > 
> > type_name: 
> >           The name under which the interface is registered with GType.
> > 
> > type_init:
> >           The symbol name of the get_type() function which registers
> > the type.
> > 
> > n_interfaces:
> > n_values:
> > n_fields: 
> > n_properties:
> > n_methods:
> > n_signals:
> > n_virtual_functions:
> >           The lengths of the arrays.
> > 
> > parent:   The directory index of the parent interface. This is only
> > set for 
> >           objects.
> > 
> > Up to 16bits of padding may be inserted between the arrays to ensure
> > that they start on a 32bit boundary.
> 
> How do we detect that is the case? And will 32bit be sufficient
> alignment on all archs?

You just count the bytes from the beginning of the InterfaceBlob, which
is known to be on a 32bit boundary. E.g. if the number of interfaces is
odd, there will be padding between interfaces[] and values[].

"Sufficient" for what ? It may not match the natural struct layout on 
the platform, but it is enough to be able to read the values from the
blob as guint32. If a guint32 value is straddling a 32bit boundary, it
may be necessary to read it as to guint16 and reconstruct it manually
on some platforms.


> > interfaces:
> >           An array of direcory indices which for objects point to the
> > implemented interfaces, while they point to prerequisited for
> > interfaces. 
> 
> Minor nitpick, I'd rephrase it as: "An array of direcory indices
> pointing to the implemented interfaces for objects, and to prerequisites
> for interfaces."  Besides, it's confusing to use "interface" for both
> GInterface and metadata interfaces.

I'd be happy for better naming proposals. 


> [snip]
> 
> > struct AnnotationBlob
> > { 
> >   guint32 offset;
> >   guint32 name;
> >   guint32 value;
> > }
> > 
> > offset:   The offset of the interface to which this annotation refers.
> > Annotations are kept sorted by offset, so that the annotations of an
> > interface can be found by a binary search.
> > 
> > name:     The name of the annotation, a string.
> > 
> > value:    The value of the annotation (also a string)
> 
> IMHO, strings are not good enough. Annotations should be real type,
> similar to objects (but simpler), with parameters and inheritance.

Well, I'm not sure this is really worth it. Do you have any concrete use
cases in mind, or is this just general nonsense, 'but Java5 has it'
stuff ?

Matthias