Re: [g-a-devel] D-Bus replacement for AT-SPI Accessibility

[Willie Walker <William Walker Sun COM>]

Hi Mark:

Nice writeup -- thanks for doing it.  :-)

> - The 'Collection' and 'StreamableContent' interfaces still need to be
> implemented. Some code has been written but is currently not built and
> has not been tested.

The Collection interface is indeed used by assistive technologies.  When 
it works, it works well and helps address performance issues.  It's not 
always ideal, though, and sometimes ends up being pretty non-performant 
(I cannot remember the exact details right now).

Collection is also sometimes insufficient to do what want, such as 
matching an item in an attribute value where the attribute value is a 
list.  An example of that is the 'xml-roles' attribute exposed by Gecko. 
 This can be a list of roles and we'd like the ability to make a match 
if there is a specific role in the list.  Instead, I believe the current 
Collections API requires an exact match for the entire list.  This is 
because the 'list' really is a string and the delineation of items is 
somewhat arbitrary: the delineator can be a space, a semi-colon, etc.

If my understanding is correct, Collection does a lot of work at the 
a11y abstraction level.  If that's the case, I wonder if moving to the 
"push the whole hierarchy to the assistive technology and cache it 
there" model might end up reducing one of the reasons for Collection.

Note, however, that an assistive technology needs to be able to discover 
items that are not painted on the screen.  An example use case is going 
to the next table in an OOo document, where the user might be on page 2, 
but the next table is on page 5.

In any case, my point is that the Collection API thoughts/decisions may 
hinge off the other decisions being made.  In one case, it remains 
something at the ATK side.  In another case, it might be something that 
ends up being an AT side convenience library.

I'm not sure of any consumers of the StreamableContent interface, but 
they may emerge.

> 1) Keeping track of all the accessible applications on the desktop.
> 2) Dealing with and forwarding X events over AT-SPI D-Bus.
>
> The registry of accessible applications is relatively simple and has
> been fairly well tested.
>
> - DeviceEvents are given non-valid D-Bus message names.
>
> - The DeviceEventController object that handles X events needs to be
> re-factored.
>
> - Possibly XEVIE code should be removed as this will not be available in
> future X.org releases.

The DeviceEventController is indeed very important for the purposes of:

1) Sniffing input events
2) Intercepting/consuming input events
3) Synthesizing input events

This is definitely a key thing that needs to be ported and I'm glad 
you're working on it.

> - All D-Bus method calls in python are synchronous and are not
> re-entrant. This can cause deadlock. Need to make a re-entrant D-Bus
> method call available in python and change all synchronous calls over to
> this.

Out of curiosity, where does the deadlock occur?  Is this something that 
can cause the entire desktop to lock or just the AT?

> cspi
> ----
>
> A client side library in 'C' for access to AT-SPI accessibility information.
>
> It is unknown how much work is required to complete this library. It has
> been working and tested previously, but has not been updated with
> protocol changes and has never been heavily tested.

So, lots of work has already been done (cool).  What's the extent of the 
protocol changes that have been made since the last changes were made to 
the cspi port?

> loginhelper
> -----------
>
> An extension for enabling accessible login.
>
> Unsure

I'm CC'ing Brian Cameron explicitly for this since he did a lot of the 
work for gdm accessibility.  Brian, can you help lend some insight into 
what kinds of issues one might expect when porting loginhelper to the 
new D-Bus infrastructure?

> The importance of this is possibly over-stated. Firefox exposes the
> entire DOM and does not have this problem. Open office exposes only the
> objects visible on-screen and therefore sidesteps this issue.

Other things to look at include OOo calc and how it exposes the cells in 
a spreadsheet.  A spreadsheet is one of the larger spaces we run into. 
In OOo calc, the parent is a table that manages its descendants.  When 
you ask for the children, you currently can get them *all*, not just the 
ones painted on the screen.

I get a childCount=67108864 for the OOo calc spreadsheet on my machine. 
 That's not infinite, but I believe it would take me over 2 years to 
navigate the spreadsheet if I spent 1 second on every cell.  Pushing the 
entire child list over to the AT is probably not a good idea, and 
pushing the just the visible items might be better.  But, the AT still 
needs the ability to get to objects not painted on the screen.

In addition, I believe Java's various list/table/tree widgets may have 
been some of the initial 'inspiration' for manages descendants.

> Multi-user accessibility
...
> Remote Accessibility

Thanks much for making the distinction here.  I think the primary use 
case is multi-user accessibility, and the typical subset of that is 
accessing system administration GUI's running as root.

> GNOME Priorities
> ----------------
>
> This is a list of thing that I personally believe need to get done for
> inclusion in Gnome 3.0.

I think the main criteria for success should be:

* We meet the deprecation goals for GNOME 3.0.

* Existing consumers of the a11y infrastructure (e.g., assistive 
technologies and testing frameworks) continue to work without loss of 
functionality and without a perceived decrease in performance. 
Preferably, we'd get an increase in performance.

* The resulting solution is something that non-GNOME folks will 
support/use/integrate.  This includes the KDE folks, the OLPC folks, etc.

> High Priority
> -------------
>
> Multi-user accessibility - Required for accessible system administration.
>
> Cspi - There has been alot of discussion about whether to complete cspi,
> or a subset of cspi. I think that there is enough programs currently
> using it that we would preferably complete it. There is no guarantee
> that LDTP will have moved over to pyatspi by the transition. If we do
> not complete cspi we will need a new 'C' interface that may be a similar
> amount of work to complete.
>
> Pyatspi - Not a-lot of work. Need to create re-entrant method calls.
>
> atk-bridge. - Need a solution for the collection interface and for the
> streamable content interface.
>
> registryd - Need to refactor the DeviceEventController, and ensure that
> without XEVIE we can meet all requirements for device events.
>
> GtkTreeView - This needs to be solved, it makes GtkTreeViews either
> inaccessible or causes us to hold lost references to accessible objects.

I think this is fair.  I'd expand the GtkTreeView problem into a more 
general problem of solving the manages descendants problem, taking care 
to handle Java's list/table/tree/etc objects, OOo Calc's spreadsheet, etc.

We also need to be able to handle the use case of being able to navigate 
to objects off the screen.  For me, a couple of the main tests will be 
Orca's structural navigation in Firefox and Orca's SayAll functionality 
in a multi-page OOo document.

> Gnome integration will be required, a jhbuild module and tests building
> AT-SPI D-Bus using jhbuild.

Do you anticipate new modules being included and old ones being removed, 
or would we end up sticking with the existing module set?  If we end up 
with the same module set, except it uses D-Bus and not Bonobo/CORBA, 
then the jhbuild integration might be somewhat automatic.

> Thanks for your attention,

Thank you!

> (Surely no-one made it this far)

I took it as a challenge.  :-)

Will