Re: ARC & GNOME [Was: How we make decisions...]

[Brian Cameron <Brian Cameron Sun COM>]

Havoc/Others:

Thanks for the historical perspective of GEP.  That's useful to hear.
Personally, I would like to avoid having a discussion where we only
talk about process and theory - so I'll inject some practical ideas
and suggestions about things we can do to improve the relationship
between the GNOME community and ARC.  My hope is that I can highlight
ways that we could more closely work together that would benefit both.

> > Ultimately, the ARC question (as viewed inside Sun) comes down to where/how do 
> > you capture interface related changes before they occur.  Once you have them 
> > captured you can refer back to them later.  That's one of the prime things of 
> > the Sun process.  Is this something you all want to get out of the GEP process 
> > or is it captured somewhere else? 
>
> The official GNOME interface policy is that we don't change the public
> interfaces. "Things intended to be public interfaces" is not as well-
> documented as it should be, but would include for example GTK+ which
> hasn't changed since 2.0. (Well, really it hasn't changed since 1.2,
> since 1.2 interfaces are still there and working -
> http://ometer.com/parallel.html)

I think that the GNOME community has done a good job of insuring interface
stability from a library ABI perspective - especially for the core libraries
such as glib/gtk+/pango/etc.  However, library interfaces are not the only
interfaces that can change and cause problems.  Other things include:

+ The way programs interact with environment variables
+ File locations, for example integration points where users or developers
  can install files and expect certain behaviors.  Also that programs are
  installed to the right location (libexec instead of bin, for example).
+ Changes to data structures, enumerations, etc. that are passed into
  functions.
+ The syntax of files that control behavior, such as GConf files, the
  files that control theming, how the component model works, etc.

For example, when Sun released GNOME 2.0 we decided to put datadir files
for modules that Sun classified as "Evolving" into /usr/share, and datadir
files for modules that Sun classified as "External" into /usr/share/gnome.
Setting up our environment this way created a bit of a nightmare for our
release engineering, QA, and developer teams since this caused a number
of bugs to appear that we had to resolve - with mixed success.  Now
that we are releasing GNOME 2.6 we decided to install everything into
/usr/share.  This makes things simpler and reduces headaches.

However, we are now finding that certain customers installed files into
the /usr/share/gnome directory (such as bonobo XML UI integration files,
pixmaps, etc.).  They are understanably unhappy about having to move
files around just because they are upgrading their system from GNOME
2.0 to 2.6.  So now we are fixing "bugs" to make GNOME 2.6 also look
in the old /usr/share/gnome directories for certain files.  However
it is difficult to know exactly which datadir directories we should
support in this way since it isn't always easy to determine which
subdirectories are private interfaces for a specific application and
which ones are reasonable public integration points.

One solution that I would like to suggest would be to enhance gtk-doc
so that it can produce more expansive interface information.  It would
be nice if gtk-doc could provide information about file integration
points, environment variable usage, changes to enumerations and data
structures, and so on.  gtk-doc currently gives information about when
a given library function was added, but it would be nice if gtk-doc also
gave more clear details about when interfaces were depricated.

In order to do our internal ARC documentation here at Sun, we typically
have to troll through the code and dig out as much of this sort of
information as possible by hand.  As you can imagine, doing things this
way is time-consuming, painful, and we typically end up with documentation
that is far from complete.  Annoyingly, the documentation we end up
creating is in Sun ARC's format, which is proprietary and difficult to
share back with the community for review or public use.

However, if we were able to work more closely with the community, we
here at Sun could do the same work but in a way that benefits the community.
Enhancing gtk-doc so that it can produce the information we need would
likely be useful to the community, since users/developers in the community
are likely also interested in the details of interface change.

It would also be useful to enhance gtk-doc so that interfaces could be
labeled with a stability level.  This way users of the interfaces
would know if the community expects the interfaces to be stable, or if the
interfaces are likely to be changed sometime in the near or far future.

In other words, there is a real interest here at Sun to enhance the
public documentation and the tools used to create the documentation.
This would save us a lot of work here at Sun and would improve the
documentation for the community at large.  My hope is that if we improve
gtk-doc to allow more interfaces to be documented, that developers in
the community will help to maintain the docs as interfaces change.

The end result should be documentation that would make problems like the
one I described above (Sun not knowing exactly which /usr/share/gnome datadir
directories that need to be supported) more straightforward.  It would
just be a matter of checking the docs, and the interface stability levels
to know which ones require backwards compatibility support.

However, I don't think that the right approach is for someone here at Sun
to just start hacking away at gtk-doc to make it produce information that
is useful for Sun.  Like I said above, such an effort will really only
succeed if there is enough community interest to help maintain the docs.
In other words, it is really only useful to add new features to gtk-doc
that the community is also interested in supporting.  So it seems a good
idea to throw these ideas out there and see if the community would be
interested in such gtk-doc enhancements and would the community help to
maintain the docs if such enhancements were added.

> > Sun currently spends a lot of time and energy trying to ensure that the GNOME
> > stack is as stable as possible via our internal process.  We think it might be
> > more beneficial to review the GNOME stack from this perspective directly with
> > the community.  If Sun were willing to work within the GEP process and were
> > able to provide resources towards improving interface stability, would this be
> > of interest to the community.  We at Sun are thinking that this could be a
> > better approach than Sun trying to address these issues independently and we
> > are wanting to know if there is also a community interest in this area.
>
> I think the community feeling is that the interfaces *are* stable.
> That's at least the intent... while accidents do happen, said accidents
> would normally be treated as bugs. If we take interface breaks as
> accidents then the solution is really QA, rather than docs...

As someone else mentioned, I think that the community focuses primarily
on library ABI interface stability, and not other interfaces.  Furthermore
the community is not very clear about which libraries are considered
stable and which ones aren't.  There doesn't seem to be any docs on
developer.gnome.org, for example, which highlight "these libraries are
stable, these ones will probably change down the road, and these ones
will change in the next release."

> I've seen a number of references to concerns about interface stability
> from Sun; I'm curious what the examples are. 
>
> My guess is that you guys are worried about interface stability in
> things we would not consider public interfaces. e.g. libwnck, or I would
> argue that the applet interface isn't ready for ISV consumption.
> In those cases, normally the solution would be writing code to bring
> those interfaces up to supportable form or resolve the reasons we don't
> feel we can commit to them.

Right, that is part of the problem.  As I mentioned above, part of the
problem is that it isn't really clear which libraries are considered
public/stable and which ones aren't.  Nor does there seem to be a community
process to "bring those interfaces up to supportable form or resolve
the reasons we don't feel we can commit to them".  Part of the problem
is that there isn't any public documentation to explains what
problems exist that would need resolution to make this leap.

I suspect such information could be found if a person spends enough
time digging through the mail archives, but there really should be a
place on the developer.gnome.org website where such interface information
is centralized so it can be used as a resource.  Right now it seems
that this sort of information is mostly captured in the heads
of the people who are most closely involved with the modules.
While this might be sufficient for the development of the GNOME desktop,
it doesn't provide ISV's with any usable resource to figure out what
interfaces they should consider using.

Another problem with ARC'ing GNOME is that the GNOME community does
not seem to have a consistant mechanism for change notification.
Therefore, we typically troll through the code and try to highlight
changes when we go to ARC review.  Since we do this process without
enough involvement of the community, the end result is typically that
we end up with an incomplete review that only focuses on certain
highlights.

If there were some mechanism in the GNOME community for developers
to notify the public that interfaces are going to change and how,
then that might be generally useful to everyone.  It would provide
a place for people within the community to review interface changes
as they are happening and provide comments or recommendations about
how things could be done better.  It would also provide a central
reference point where people could go and see what is changing
over time.  This could be achieved by setting up a mail alias
where people who are changing interfaces send an email to let the
community know about change as it is happening.  Also, the GEP
process states that it can be used for reviewing interface change,
though I don't think GEP has been used for this purpose so far.
Simply making use of the GEP process in this way might be a way
for the community to provide more notification about change.

If such a process could be set up, then Sun's ARC is very
interested in participating in a review process.  They are very
skilled at noticing the sort of stability problems that happen
when interfaces change, so I think getting them more involved
with the GNOME development process would help to make GNOME more
stable.

> I've been saying for years that libgnome/libgnomeui are busted and
> shouldn't be recommended to ISVs; that's now being officially
> acknowledged with the deprecation plan on them. But the interfaces won't
> be broken, only deprecated, so if some ISV does use them they will keep
> working.

I've dug around on developer.gnome.org and searched around the mail
archives, but don't see where the deprication plan is published.
Could you point me towards the plan?

> Another area of work would be tools such as Sun's abicheck or the LSB
> tools for verifying that ISVs use only supported interfaces...

Yes, I think that there are definately opportunities for Sun and the
GNOME community to work more closely sharing tools that can be used
to better ensure interface stability from release-to-release.

--

Brian