Re: Comments on GTK+ patches from Atk tarball

[Bill Haneman <bill haneman ireland sun com>]

Hi Owen:

I have tried to address your concerns in detail, to provide some more
justification for our design.  I'm afraid my reply is rather lengthy. 
In summary, and to paraphrase:

Q: Why expose GtkAccessible ?
A: Arguably it can provide a little insulation from ATK for the GTK
programmer, if we add some convenience methods.  Also, the custom
widget maintainer who chooses to create AtkObjects without a factory
(one of several options) will probably want to subclass GtkAccessible
instead of AtkObject.  

Q: How do custom widget maintainers use this API?
A: There are at least 4-5 major variations - implement ATK interfaces
directly, subclass the factory, add new factory implementations to the
library, etc.  (see below)

Q: Why keep non-ref-counting "get" accessible instead of uniform "ref"
?
A: Accessible objects are usually _not_ flyweights, the only time you
might get flyweights are when using the "ref_child" methods which
applications and GTK internals will not need to do.  We want to make
it as simple as possible for applications to use this API, and
limiting flyweights to the "ref_child" methods allows us to do this
consistently.  We feel strongly that we need this model rather than
using "ref" everywhere.

Q: What's the deal with two ways of getting a widget's accessible ?
A: App and GTK maintainers will never use the cumbersome
AtkAccessibleIface entry point, that's there solely for the use of
bridge code and other customers of ATK which need a uniform,
non-GTK-specific entry point.  Consider these to be analogous to two
separate "bindings", only one of which will be used by GTK+ and
GTK-apps.  Again, we think that both are needed since they satisfy
separate design requirements.

Q: Why use factory objects and not function pointers ?
A: Factory objects may need to keep state or data, particularly if
they use cacheing, hashing, etc.  The object pointer needs to be
available to the GtkWidgetClass because we will sometimes need to
query the factory type (for instance, in order to decide whether the
factory needs to be replaced if an assistive technology wants to
install its own specific factory type).

Q: Are factory objects really a good way of providing accessibility
implementation for GtkWidget subclasses ?
A: They are working well for us so far :-)  They allow us to map
accessibility support features onto the GtkWidget hierarchy in a
general way, so that the inheritance graphs do not need to coincide
and we can provide the greatest coverage for the widget hierarchy (via
inheritance of the factory instances, and inheritance within the
separate GtkAccessible hierarchy) with the minimum amount of code
duplication.

Best regards,

- Bill

Owen Taylor wrote:

> > >   Basically, why have a publically visible gtk-namespaced AtkObject
> > >   derivative? People using the accessible functionality will be
> > >   referring to accessibles as AtkObject, I assume.
> >
> > Actually, we anticipate that users of the "set" methods within GTK will
> > refer to GtkAccessible instead of AtkObject.
> 
> What's the practical difference? The users will be calling methods
> on different ATk interfaces, right?

Functionally there isn't much difference, I agree.   However - in the
implementation itself we have a hierarchy of subclasses of AtkObject,
which descend from a common ancestor class, GtkAccessible.  The issue,
as I mentioned already, is whether to expose this ancestor, or to
expose only AtkAccessible.  

> > At the moment, no, we only
> > see the need for the back pointer, but the back pointer *is* important,
> > the only question is whether this class should be visible to GTK+ or
> > hidden in the implementation library.
> >
> > We see two reasons for exposing this class:
> >  1) we may need to add to it later, without breaking binary compat;
> 
> There are a lot of other things you might also want to add later...
> Actually, extending this class to do anything useful will probably
> break binary compatibility.

I mean binary compatibility with GtkWidget, not GtkAccessible.  But
keeping GtkAccessible, and using it instead of AtkObject as the return
type for gtk_widget_get_accessible(GtkWidget *), helps preserve source
compatibility going forward, which may be just as important.  Binary
compatibility within Atk may break a couple of times on the way to the
2.0 library milestone, that's no problem since nobody is relying on it
yet, but we don't want any _source_ incompatibilities after next
month. 
 
> Can you expand a little on why you think you might want this class
> in the future?

We need it because we need the back pointer to GtkWidget, which is not
generally used by AtkObject but is required by the GtkWidget-specific
implementations.

I agree that it would be possible to use it without exposing it (we
could keep it in the implementation library and rename it to something
else).  But there are other advantages to exposing it (below)...
 
> >  2) it is needed (or at least very helpful) for some of the several
> > approaches to custom widget support which we envision.
> 
> Can you describe these approaches?

I mentioned three ways in previous emails, here is a short summary:

A custom widget maintainer can:

(1) Implement AtkAccessibleIface on the custom widget directly, and
override GtkWidget::get_accessible to return a subclass of AtkObject
whose state is maintained by the widget.  In this case the dynamic
runtime library is not used by this widget, the custom widget takes
over all responsibility for accessibility support.  One possible use
case for this would be ab application that wanted to work with the
standard accessibility module but for some reason did not want to
publish its source or accessibility support code.

(1a) The widget can ignore the factory and create a subclass of
GtkWidget which implements accessibility - cleaner than (1) and more
code reuse.  The backpointer to GtkWidget in GtkAccessible is likely
to be useful in this scenario, another reason for exposing it in GTK
and not hiding it in the dynamically loaded library.

(2) The custom widget can accept the behavior from the
GtkAccessibilityFactory inherited from its parent class.  In those
cases where the widget's customizations break this behavior, the
widget can explicitly set the appropriate accessibility properties on
the factory-instantiated accessible object; for instance, a custom
button that did not use the standard "label" property for its textual
representation would need to explicitly call atk_object_set_name()
with the appropriate string on instantiation of the GtkAccessible and
every time the label state changes. (Note that accessible "name" means
"component name/label" and not "component class name").  This will
probably be the easiest approach for many custom widgets, and requires
no changes to the accessibility support library.

(3) The custom widget maintainer, or the maintainers of the
accessibility runtime library (Padraig and I, in other words ;-) can
register a new GtkAccessibilityFactory for the custom widget type. 
This is the way the custom widgets in Gnome-libs, etc. will be
handled.  It also means that support can be added without altering the
library containing the custom widgets.  If an application doesn't have
the latest runtime accessibility library then the custom widgets will
just inherit a parent class's accessibility behavior.  

(3a) Alternately, a custom widget initialization class could itself
register a new factory with the runtime library, if present - but this
would introduce a dependency between the custom widget code and
libagtk, which might not be desirable. 

There are other permutations, mostly stemming from the fact that there
are several places in the GtkAccessible instantiation chain where
default behavior can be overridden:

GtkWidget->get_accessible [ virtual function, can be replaced ]
GtkWidget->access_factory [ different factory can be installed ]

GtkAccessible behavior can be modified/extended via subclassing of the
GtkAccessible descendant which the factory creates (that's the normal
distinction between different subclasses of GtkAccesssibleFactory,
they return different subclasses of GtkAccessible).
 
> > As for disadvantages, well, it doesn't seem harmful.
> 
> All API has a cost:
> 
>  - Makes the documentation and system harder to grasp

I think that in this case GtkAccessible provides a does not create a
barrier to documentation or understanding, it may even be easier since
it provides a measure of insulation between the Gtk namespace and
Atk.  At the moment in order to programmatically use GtkAccessible one
must resort to the atk_* methods, but we have previously suggested
adding convenience methods such as gtk_accessible_set_value() and
gtk_accessible_get_selection_start(), etc. so that an application
maintainer need not even read the atk headers.  I personally think
that this would be worth the additional bytes occupied by the larger
headers.  The one fly in the ointment is that the child-reference
methods will need to return AtkObject* rather than GtkAccessible*
unless we use GtkAccessible for our flyweights, with null GtkWidget
pointers.  A tradeoff, admittedly.

>  - May interfere with future changes

It's arguable that exposing GtkAccessible will make future changes
easier.

>  - Slows things down / increases disk space (small for any one
>    addition, but accumulates)
> 
> Once we add something to GTK+, it is, for all practical purposes
> permanent. (Timescale for removing is a number of years.)

That was one reason for creating the extra level of insulation between
Atk and Gtk, 
so that GtkWidget included gtkaccessible.h, thus only indirectly
referencing atk.

> > I think there is a misunderstanding regarding get_accessible, as
> > evidenced by the two comments below:
> 
> I think there really isn't a misunderstanding (or at least,
> I figured out what you were doing at some point while writing
> the mail), but rather, my gut instinct is that trying
> to have your cake and eat it too for ref vs. get is wrong.

If we go with flyweights for everything then we will introduce large
burdens in terms of how the interfaces are used, it will make the use
of the API by app maintainers much, much uglier.  And we don't need
flyweights for everything - we really need persistent objects!  But in
the minority cases where we *do* need flyweights (these cases occur
only when querying for children) we must provide for them.  

I think it's very straightforward from the point of view of the Gtk
developer - gtk_widget_get_accessible() always yields a persistent
object, which can keep its hard-earned state and which the caller
isn't burdened with managing.  Only the methods that return children
of accessible objects can return flyweights, there is a clear
distinction between gtk_widget_get_accessible (GtkWidget *widget) 
and 
gtk_widget_ref_accessible_child (GtkWidget *widget, gint i)
 

Now, the one other complication is the "2 points of entry" thing,
where there is a "ref accessible" and a "get accessible" - that's a
consequence of having a generic accessibility interface
(AtkAccessibleIface) as well as a get_accessible() on GtkWidget.  Use
of the interface API is much clunkier and certainly not something we
want an application maintainer to have to do, however we have the
requirement of having an API that can be implemented on non-GTK
widgets. There's nothing wrong with having two entry points, it's just
like having more than language binding, different uses/contexts need
slightly different interfaces in order to maximize both flexibility
and ease of use.  The ordinary user of GTK need not give any thought
to the AtkAccessibleIface implementation in GtkWidget, it's only there
for compatibility with (anticipated) other Atk clients.

> > > * The fact that there is a public gtk_widget_get_accessible()
> > >   doesn't strike me as right. I would expect simply people
> > >   to use atk_object_ref_accessible (widget). [ There are
> > >   some naming problems in Atk with the interfaces, etc,
> > >   I'll comment on separately. ]
> >
> > Within GTK+, the accessibles are *not* flyweights (unless they are
> > returned from one of the "ref accessible children" calls).  The normal
> > use within GTK+ and also within the implementation library is to use
> > gtk_widget_get_accessible(), which does not return a flyweight and
> > therefore relieves the caller of memory management worries.
> 
> > The "ref_accessible" variant is there only to provide compatibility
> > with the alternate, more generic, AtkAccessibleIface interface.
> > This more generic interface (which can be implemented on objects
> > that are not GtkWidgets) will most likely only be used in the
> > "bridge" code between ATK+ and the out-of-process SPI that is used
> > by the assistive/adaptive technologies themselves.
> 
> As far as I see, you aren't mirroring the ATK interfaces into
> GTK+. So, someone using ATK for acessiblity-enabling a GTK+ program
> will be using large portions of the ATK interface.

As I said already, it might be a good idea to do this mirroring, in
the form of convenience macros.  What are your feelings?

> As far as I understand it, ATK treats all accessibility objects potentially
> as flyweights, and all navigation functions that return another
> AtkObject return a new reference to that object?

I think of the flyweights as exceptions rather than the rule.  From
the application maintainer's perspective, very little use is likely to
be made of the child-reference methods; however the bridge to the
Assistive Technology SPI is likely to use these methods heavily.  

For instance, an application may often use something like this
(ignoring convenience macros, for the moment):

GValue value;
gboolean val_was_set;
...
/* set the GValue */
AtkValue accessible_value;
AtkObject accessible = gtk_widget_get_accessible (widget);
accessible_value = atk_object_get_value (accessible);
val_was_set = atk_value_set_current(accessible_value, value);

The app will rarely, if ever, want to do a "set" on a widget
accessible's child  - partly because if it's a flyweight the change
will not persist!  So the app maintainer will not be concerned with
the "...ref..." methods.  On the other hand a custom widget maintainer
will need to implement the ref methods in some cases (see the previous
discussion about custom widget support).  

The assistive technology will be needing child enumeration a lot, but
the headache of managing the ref counts will be contained to the
"bridge" to the SPI.  We only have to write one bridge... but we want
to make it as easy as possible for apps to use the API in the way they
need to, even if it means that the bridge maintainer must be careful
about "get" versus "ref".  

As an aside, the above code (which omits various type checks which are
needed) is a good argument for convenience macros I think - it would
be nice if the single

val_was_set = gtk_accessible_value_set_current (widget, value);

could be used instead of the four lines of code shown.  I have no
doubt that the degree to which this API is used by app maintainers
will have an inverse relationship to its verbosity...

> 
> If so, it seems that someone who wants to use ATK with GTK+ needs
> to learn:
> 
>  AtkObjects are (potentially) flyweights and are ref'ed on return

Nope, only someone using AtkAccessibleIface directly needs to know
this.  But GTK+ users should be aware that the atk_ref_child
(accessible, i) [or, if we put in the convenience macros,
gtk_widget_ref_accessible_child (widget, i)] methods do update
reference counts since the objects they return are "potentially"
flyweights.  However ordinary GTK+ users should rarely, if ever, need
to use these methods.

> 
> While this may be painful, it's easier to learn than:
> 
>  AtkObjects are (potentially) flyweights and are ref'ed on return,
>  except that there is a special AtkObject associated with each
>  widget which is not a flyweight.
> 
> Have you thought more about how the new tree/list widgets fit
> into the picture?

Tree widgets are always a little harder.  But, again, there are two
perspectives: from within the widgets, for instance within each list
item, one uses get_accessible to access the accessible
representation.  From within GTK or the app, one only needs to deal
with "ref" when the children do not have associated widgets.  I need
to look more closely at the new tree widget before I say more, I am
not sure whether all tree nodes are GtkWidgets or of a GtkTreeView can
have flyweight nodes...  GtkClist seems straightforward.  Which widget
is the preferred widget for lists now?  I thought GtkClist was
deprecated but I am not sure of the replacement, should one use a
special case of GtkTreeView ?


One thing to keep in mind, from the app maintainer perspective, and
from within GTK+, only the "set" methods will be used (at least 99% of
the time) - with the sole exception of entry point  "get_accessible". 
The other "get" and "ref" methods, which are where this issue seems to
live, are required by the implementation library and the assistive
technologies (via the SPI).  The balance we are striking is this: 
Make it really easy for application maintainers and GTK+ internals,
and shift the complexity to the implementation library where it is
necessary.

> [...]
> 
> > > * While GtkAccessibleFactory makes sense, it strikes
> > >   me that simply using a function pointer for this would
> > >   be simpler / easier. Using a class for this is (IMO)
> > >   overkill, and we certainly use function pointers througout
> > >   GTK+ for similar things.
> >
> > Again, we think that using a class is more extensible.
> 
> The class is more extensible, if extensibility is needed. It also is
> harder to remove if it turns out we needed something altogether
> different.

I don't really follow how it will be difficult to change the factory
implementation if it's a class pointer instead of a function
pointer... if it's a class pointer then we can add all sorts of stuff
without disturbing the widgets at all, just the same as if it were a
_pointer to a function that's inside the class_, which would be the
alternative.  More below...

> I'm not horribly opposed to GtkAccessibileFactory, but it is different
> than all the other places in GTK+ that we do something like this,
> so I'm wary of it.
> 
> Wrapping function pointers as classes makes a lot of sense for Java,
> and (to a lesser extent) C++. It makes less sense for C and for
> languages like Perl, guile which have first-class function objects.
> 
> Basically, my feeling here is that the accessible-factory is something
> that is only important to the accessibiltiy-enabling library.  As far
> as a I can see, it isn't something you would use when implementing
> your own custom widgets.

As I mentioned above, one of the nicer ways of doing custom widgets is
subclassing an existing  GtkAccessibleFactory and installing it into
the custom widget class struct.

> So, keeping the API here small and simple is more important to
> me than keeping it cross-language, generic, extensible, etc.
> 
> > It also means that we can implement reflection techniques on the
> > factory instances after they have been associated with widget
> > classes, which would be difficult if not impossible with function
> > pointers.
> 
> Could you explain what you mean by this?

It turns out that ATs will sometimes insist on doing
application-specific things with widgets, the cleanest way of doing
this in our current architecture is to allow the ATs to replace the
accessible factories with specialist versions - at which time new
accessibles for the widgets would be created, copying any
pregrammatically-set values from the existing persistent accessible
objects, and then the old accessible objects would be freed.  I am not
so keen on the idea of doing application-specific stuff, but the AT
vendors that I have engaged in conversation about this have been very
insistent on this capability.

One consequence of this is that factories may be replaced - and so
before replacing a factory instance we need to make sure we are
substituting an instance of class B for class A instead of just
replacing one B with another... so we need to be able to find out, for
a given widget class, what its factory instance type is.  If we just
use function pointers that will be very messy, won't it?  Or it there
some hocus-pocus with static variables that we could do?  

It still seems much more OO to use classes (structs).

 
> > > * _gtk_weaktype_ref_accessible shouldn't have the leading
> > >   underscore, since we don't use that for static functions,
> > >   and by GTK+ convention, would be simply gtk_widget_ref_accesible.
> >
> > OK.  We just wanted to emphasize that this method is private.
> 
> Not exporting it from the C file is enough emphasis for that...

<grin>
 
> > > * The overhead of initializing every class in GTK+ is quite
> > >   large and we intentionally delay initialization until an
> > >   instance of the class is created. So, I don't think
> > >
> > >    gtk_widget_class_set_accessibility_factory()
> > >
> > >   is a good idea; better to have a global factory.
> >
> > There isn't a global factory, there are many factories.  Different
> > factory instances must be associated with different classes, and
> > changeable at runtime, thus the method above is required.
> 
> Can you provide more background for these statements?

I've tried to, above.  Maybe we should try IRC ?  I'm  behind two
firewalls today, maybe Monday ?
 
> This seems to mostly be an implementation detail of the
> accessibility-enabling library;

Yes, and we could move the whole factory/registry thing to the
library, except:

(1) the initial AtkObject/GtkAccessible instantiation would have to go
through a lookup, once for each widget;

(2) it would limit the ways in which custom widget maintainers could
add support, since they could only override the (inherited) default
factory by modifying the runtime library.
 
> And if I was writing a library in C to enable accessibility
> in GTK+, I don't think I'd find creating a separate factory
> class for each type of widget I wanted to enable particularly
> convenient.

On the contrary, we are finding it very convenient since we can
inherit not only the factory but behaviors of the factory (via
subclasses of the factory itself).  In short we have a hierarchy of
factories and GtkAccessible subclasses which has a mapping (but not
1:1) onto the GtkWidget hierarchy.  This is, by the way, more flexible
than implementing accessibility interfaces on the widgets directly
since the most effective inheritance graph for accessibility behavior
may not match the graph for widget inheritance itself.  Of course the
direct-implementation approach was rejected early on for other
reasons, to do with impact on GTK+, GTK schedules, etc.

Here's an example:  by creating a GtkWidgetAccessibleFactory whose
GtkAccessible instances (GtkWidgetAccessibles) implement AtkComponent,
we can register this factory on GtkWidgetClass and get AtkComponent
behavior for the whole GtkWidget hierarchy.  If we subclass
GtkWidgetAccessible for GtkContainer and GtkButton, we can provide
increasing levels of coverage for the whole GtkWidget hierarchy with
only a few accessibility implementations.  Our guess is that for ideal
coverage we will need somewhere around 30-40 different subclasses of
GtkAccessible, perhaps we can get by with fewer.  Most of the work
will, in all likelihood, go into things like implementations for
GtkTextView and GtkTreeView, and some custom widgets may require a
fair bit of work as well.

> Regards,
>                                         Owen

-- 
--------------
Bill Haneman
Gnome Accessibility / Batik SVG Toolkit
Sun Microsystems Ireland