Re: GConf and bonobo-conf

[Colm Smyth <Colm Smyth ireland sun com>]

>> If GConf supported CORBA_Any at the interface level, it would be
>> widely used because it would be simpler and easier than other
>> solutions; absolutely no doubt there.
>
>	Ok; so you might approve generaly of the CORBA_Any if it was
>properly integrated into the GConf API and the evils of ConfigValue went
>away ?

Not just the API, the underlying data model is at fault.

>> The problem is that the GConf API was not designed to handle
>> self-describing data; although you can use GConf to write a CORBA_Any
>> (as XML, which is great), the interface is pretty crude since you
>> write it as a string. This is basically turning GConf into a logically
>> infinitely large indexed file and giving you an API consisting of
>> read() and write().
>
>	Yes, we logicaly turn gconf into a very strange ( string seeked
>) read / write medium. It has some of the more useful advantages of gconf,
>such as clever backends / mapping of the configuration space ( as far as I
>understand ), but yes, 100% correct.
>
>> For the application that creates the configuration data, this is
>> heaven; you can write your data-structures and read them back easily.
>> For any other application, or a different version of the same
>> application that has a different version of the data-structure, this
>> is bad news.
>
>	Yes, again - this is true only if gconf refuses to do the sensible
>thing and build the CORBA_any into the API, so there is no scope for other
>applications to get confused. This is another reason why a single - bonobo
>/ CORBA_Any based API makes so much sense - there is no half way house
>for apps to poke around and get confused at.

As I've mentioned before, building CORBA_Any into the API doesn't help
other applications interpret the data, access/modify discrete data 
elements, 
or deal with data versioning.

>> Some API's, such as DOM, are designed to handle self-describing
>> data.
>
>	I read a load of the stuff on the w3c site about the DOM, but I
>didn't see specific mention of self-describing data - CORBA_Any's have
>very explicit, well defined semantics for their introspection. This
>coupled with eg. the standard DynamicAny interface gives any application
>access to the arbitrarily complicated contained data via a standard API.

The DOM is just the API, the data model is XML - self-describing data.
GConf's data model is a hierarchy of primitive values.

XML allows you to store binary data, but then it ceases to be what
everyone expects XML to be (self-describing metadata-rich data).
Storing an encoded IIOP value in XML using a non-standard encoding
would not make your data available to apps that expect you to use
XML as it was intended.

>> This is fine, but it means that I can't use any of the power of SQL to
>> query that data or perform fine-grained updates.
>
>	I take the fine grained updates point, but then balance that
>against the app developer having chosen this structure, and the huge
>round-trip penalties we are talking about for repeated single key
>fetching.

Performance is a different issue, partly dealt with already using
gconf_engine_all_entries(). I proposed an API to allow a list of
arbitrary keys to be returned in one call, this can be easily
achieved using the existing infrastructure and it works within
GConf's data model.

>> Other applications can't easily view or update the elements of my rich
>> data.
>
>	This again is obviated by the pluggable, expandable bonobo-conf
>TypeCode <-> Control editor mappings, allowing exotic types to be edited
>in possibly 2 ways:

I responded to this point earlier; apps that need programmatic access
to the configuration data aren't helped by a GUI-editing framework
(although this is a really useful extension, somewhat similar to
the JavaBeans model where an IDE can use JB-specified editors).

>	* Plain old boring - simply show the structured data as it is
>	using primitive type editing.
>
>	* Pretty exotic - eg. a Bonobo/Color produces a color selector
>	control.
>
>> If you write a rich data-structure as a string to GConf, here are the
>> disadvantages:
>>
>> - other apps have to "know" that this string is an XML document
>>   corresponding to a specific data-structure and either manually
>>   parse the DOM and perform an update (but how? attributes or
>>   tags?) or know that this XML document represents a specific
>>   data structure that could be converted back to a CORBA_Any
>
>	Yes this sucks, however mandating the storage of richly typed
>data, and removing the 'half way house' C API removes this issue.

That means that all apps have to work with CORBA_Any; how do I write
an app to interpret this data unless I have access to the details of
the data-structure that the owning app used to write it? I'd have
to either include your header file (creating a dependency so that
every time you change that data-structure in your application, I have to
re-build mine) or parse the CORBA_Any manually (not easy code to write)
using knowledge of what you intended the structure to be.

>> - if the data-structure evolves, then the ability to transform
>>   the XML from a previous version of the data-structure is lost
>
>	I do not believe this needs to be the case, any more than is so
>than with gconf's structure currently. Conceptualy it is extremely obvious
>to me that if we have a type description:
>
>In the DB:
><types>
>	<type name="a" value="long"/>
>	<type name="b" value="string"/>
></types>
>
>Requested Type:
><types>
>	<type name="b" value="string"/>
>	<type name="c" value="double"/>
></types>
>
>	It is no trouble to discard a, keep b and return c as a default
>value when fetching the config data.

You seem to be suggesting that I should not use the CORBA_Any interface
but instead manually interpret the stored XML. I don't think this
requires less code than storing the individual data elements.

>> This seems to be saying "I'll gladly pay you Tuesday for a hamburger
>> today" ;) or to put it another way, "I'll choose a quick solution
>> today because I know that I can write a hard solution tomorrow".
>
>	Not really, I was answering the objection that this is not
>expandable / compatible in the same way across versions / structural
>changes - and I believe I have demonstrated ( to my satisfaction ) that it
>can be so. 

Maybe you could write some example code that uses the CORBA_Any
interface to write a rich data-structure in the way you intend and
then write the code to interpret that data when the version of
the data-structure that was written is not the version you
expect (and so you have to manually parse the stored data and map the
individual data elements to the new structure). If it's as easy as you
suggest, that wouldn't take long and would be very convincing proof.

>> I don't think that it's so easy to write code to parse the
>> data-structure and migrate the data members to an updated
>> data-structure, in fact I'm sure it's harder than writing the
>> "serialization" (configuration-saving) code using GConf primitive data
>> in the first place.
>
>	Well, quite possibly. However, since we have all the neccessary
>information we only have to write this code once for every user of the
>system. Thereafter all users can transparently, trivialy and efficiently
>map their structures beautifuly easily.

Well no, to be complete we have to write the code once for every time a
data-structure changes. To be really complete, we have to handle
bi-directional conversion from every version to every other version.
If the underlying data model could handle additions transparently,
then most version-related issues could be handled. The main issue is
ensuring that the data has the same meaning (e.g. storing an enumerated
value is basically fine, but if you like to keep your enum members
in alphabetic order, the next time you add a new member you are likely
to break old configuration data). This issue is true whether you store
your data as a BLOB (from the perspective of the data store) or as
discrete addressable data elements.

>	This contrasts with a lot of replicated, slow, potentialy buggy
>code to manualy map the values across. This seems feeble to me, if we have
>the capability to represent the situation genericaly, why throw the
>representation away as we compile hand written converters ? [ this is very
>reminiscent of the ORBit IDL compiler's view of the world - broken 8].
>
>> I am sure it's incredibly cool, but it's some cool technology on top
>> of a system (GConf) that was designed with different goals in mind
>> (easy to learn, easy to develop, easy to use); it was not designed to
>> handle rich data.
>
>	Fair enough, I take your point it is ugly having this
>multi-layered API that allows apps to see rich data as strings - it
>sucks. But I don't see that as a reason for not fixing GConf to store
>rich structured data in future revisions.

Michael, I'm really on your side; I would love to be able to easily
store rich data in GNOME's configuration system, but there is a
mis-match in the data model. If we add the bonobo-conf feature to
GConf, it becomes very difficult to persuade developers that they
should only use it for configuration data structures that will not
change (because handling multiple versions of the data is hard).

Someone is sure to suggest that we version the configuration data; I think
that's a bad idea.  Imagine you're a system administrator and you have to
migrate a lot of users from one version of GNOME to the next (say 2.1 to
2.2).  Some of the applications have some enhancements that required adding
a few data items to their configuration data, which was done by adding some
data members to a few configuration data structures. How are you going
to migrate all of the user's configuration data? What if those users
work with different machines at different times (each of which has
a slightly different version of GNOME) but they want to use
their configuration data? (should you make multiple copies of the
data, one tailored for each of the versions of GNOME? how do you
keep them in sync?) Bang! (that was a new generation of sysadmins
taking the easy way out, in the manner of stock brokers when the
Great Depression struck).


>> You didn't include the code that would be needed if you decided to add
>> a new data-member to your structure; that is the part that's
>> problematic when GConf can't interpret the opaque (but rich to the
>> originating application) data that's being stored in it.
>
>	This is shared code, written once in gconf / bonobo, yes it is
>relatively non-trivial, but it is centraly maintained and can be made
>bug-free.

Similar code is needed for every data-structure that changes, each
time it changes.

>> If GConf was *slightly* different, it would be easier; the problem is
>> that GConfValue expects to own the actual data.
>
>	A very similar problem to that already solved by CORBA_Any or
>CORBA_sequence, that of the ->_release field.
>
>> DOM is a very low-level API for XML documents, which you can find
>> documented in many places but you could start with www.w3c.org. I
>> expect that a web search-engine and the incantation "dom tutorial"
>> would also be a good path to enlightenment!
>
>	Thanks, I read some of this and it was most interesting although
>rather disappointing.
>
>	In conclusion, I'm confused as to what you are suggesting, whether
>the rich data storage is not a priority for gconf, whether the current
>setup (of being able to see rich data as strings) is broken (true), or
>whether you are set against the whole idea of storing rich configuration
>data (perhaps on the grounds of compatibility).

Actually, I've mostly been trying to point out why I think that
bonobo-conf will create heartache in the long run, I haven't
really been making a specific proposal yet. Since you ask, I
would like to:

1. Fix GConf's APIs to be more efficient to use and run, but working
   within it's data model; use this API widely and effectively throughout
   GNOME 2.x and live with the minor extra work when first writing
   code to get/set structured configuration data because it will make
   life easier for developers, users and sysadmins for all of the
   steps from 2.0 to 2.9.999999.

2. Start work defining a rich configuration API for GNOME 3.0 that
   solves all of the problems that are deeply embedded in GConf by
   using XML as a visible data-model but storing only primitive
   data at the lowest level and allowing arbitrarily nested data elements
   to be directly addressed with a single key.
   
   This API would allow an application to get/set arbitrary
   collections of GConf data (either a portion of the GConf database
   as an XML document or a single primitive data item). For efficiency,
   this configuration system or the application developer can decide
   to transmit a tree of GConf data as an XML document, leaving GConf
   to de-compose (or flatten) it if necessary and allowing the
   developer to access individual data members using the GConf API.
   
   The application-side XConf (just a codename) code could for example
   choose to fetch a tree of keys (e.g. /apps/nautilus) as a single XML
   document, but the application developer could use a simple
   API such as xconf_get_string(ch, "/apps/evolution/this/config/item").
   
   If the developer wished to indicate that they wish to work especially
   within a single sub-tree, they could use
   
   ct = xconf_context_create(ch, "/apps/evolution");
   do_that_funky_stuff( xconf_context_get(ch, "this/config/item") );
   
   If the developer wanted to write the data out as XML, they
   could do something like:
   
   xcdom = xconf_context_dom_create( ct );
   call_function_that_expects_dom_and_writes_xml( xcdom );
   
   Note that XConf internally would probably use a decorated DOM
   as an application-side cache, so the _dom_create() API would be
   trivial (just incrementing an internal reference count).
   
   A subset of the API would be appropriate for use as a CORBA
   interface; with distributed access you don't want too many
   round-trips, so one layer of XConf would be in-process on
   the application-side (for making an XML document appear as
   a collection of individually addressable data elements).
   
Item 2 naturally will take a little time, but having given this some
thought I'm confident it's a good long-term approach. Making XML
explicit in the API provides a rich data model that goes end-to-end
throughout the configuration system.

I'm looking at using some of the GConf infrastructure at least
as an intermediate step, but the API would radically change 
so no line of code above the CORBA interface would remain.

Given the proposals I've given earlier in this thread for how we
could improve GConf to solve some of the most urgent current issues,
do you think you could wait for GNOME 3.0 (maybe 1.5-3 years out)
for XConf?

Colm. 


>	Mail's rather long now, anyway, thanks for your comments, I await
>your reply eagerly,
>
>	Regards,
>
>		Michael.
>
>-- 
> mmeeks gnu org  <><, Pseudo Engineer, itinerant idiot
>
>
>_______________________________________________
>gconf-list mailing list
>gconf-list gnome org
>http://mail.gnome.org/mailman/listinfo/gconf-list