Re: [g-a-devel] Coming to grips with the state of a11y in GNOME

[Piñeiro <apinheiro igalia com>]

On 11/03/2011 09:13 PM, Matthias Clasen wrote:
> I've been asked to present my thoughts on accessibility in GNOME,
> again. After some discussions and some playing with orca in GNOME
> 3.2.1, here's what I have come up with. Again, this is largely my
> opinions, and not meant as an attack.

First, thanks for the review.

>
> The current state of accessibility in GNOME
> ----------------------------------------------------------------
>
> Most of the accessibility features of GNOME 3 can be used without
> turning on toolkit accessibility:
>
> - High-contrast and large text are implemented via theming
>
> - Zoom is entirely implemented in the shell
>
> - The screen keyboard is implemented in the shell
>
> - Visual alerts are implemented with a combination of AccessX,
>   XKB and the window manager
>
> - The keyboard accessibility features (sticky/slow/bounce/mouse keys)
>   are all handled entirely on the X layer
>
> Most of these features work reasonably well, if somewhat limited (zoom
> and the screen keyboard could clearly benefit from more widget-level
> information, such as focus tracking and entry type hints). The only

FWIW, Joseph plans to provide focus tracking for the magnifier, and also
inverse/high-contrast/brightness effects. Those were proposed as
features for 3.4

>
> The screen reader, orca, clearly is our 'flagship' AT. In my recent
> experience with orca, it worked surprisingly well and spoke to me for
> hours. The impression I got was much better than I had expected. In
> earlier attempts (sometime during 3.1.x), it randomly stopped speaking
> to me and I was unable to make it speak again.

During 3.1.x Orca had some unstable releases and required a lot of work,
mainly due last changes on static and gobject-introspection based
bindings. At some moment porting to gobject introspection based bindings
became mandatory (compared to previous "highly advisable"). Luckily
(thanks to the work made by Joanmarie) that port was finished (although
there are some regressions yet, AFAIK).

>
> Problems I've seen today include
>
> - gnome-shell crashes when orca is restarted
>
> - interacting with treeviews while orca is running easily crashes
> (just select a file in the file chooser in any app...)

Just a question, what is the application that crashes? Orca or the
application with a treeview?

>
> - people have also seen gnome-terminal crash with orca

I don't know the details, but I think that Mike Gorse already detected
and started to work here.

>
> - Insufficient labelling in new UIs, e.g gnome-shell is largely just
> 'panel' to orca...

My fault. Basically my Clark Kent job and other things sucked all the
time that I planned to use on GNOME Shell. I will try to change that for
3.4. In general GNOME Shell UI is exposed, but for most of those is
required to work on the details, to have a proper interaction.

>
> - Lack of a11y support in 3rd party widgets (stock widgetry in
> evolution is read just fine to me, but I don't hear any of the
> 'interesting' things: subjects, senders, email bodies...)

Evolution had a lot of accessibility issues for years. Most of the Orca
users prefers thunderbird (this is my conclusion taking into account
orca mailing list).

Anyway right now evolution developers started to work intensively to
port Evolution to WebkitGTK, that could improve things. At this moment
ATK support for WebkitGTK is better that the one at GtkHtml.

>
> - I couldn't get a word out of firefox - I thought maybe my gtk2 a11y
> stack was misconfigured, but openoffice did speak...

Already answered by others.

>
>
> What are the problems ?
> -----------------------------------
>
> Toolkit accessibility (ie the loading of atk-bridge into GTK+ and
> Clutter applications, as controlled by the toolkit-a11y gsetting) is
> too broken. We cannot turn something on by default that lets you crash
> any application in the file chooser. Turning it on by itself is mostly
> harmless, but as soon as ATs are actively using it, things start to
> fall apart.

At this moment at-spi or the bridge will not do anything extra unless an
AT is listening.

>
> We've tried to improve things by merging the gail module into GTK+
> proper in GTK+ 3.2. But this effort hasn't gotten us to a place where
> we can just enable a11y. The biggest remaining problem is the tree
> view a11y code.

IMHO, one of the problems here was that initially the tree view a11y
code mostly required to re-do some of the treeview logic. GailTreeView
is really big, and hard to maintain. This was required because the
treeview-a11y object has only access to the treeview public interfaces.
Now that gail was merged, ideally the tree view a11y code could be
simplified. Anyway, this is still a really hard task.

>
> There are multiple reasons why toolkit accessibility has been
> problematic ever since. Most of these have been discussed on this list
> in the past [1][2].
>
> - atk is a big interface (the scope is essentially 'export the entire
> widget tree, including all details of everything') and it contains
> things that just don't belong here, such as key event filtering

Key event filtering was also mentioned on ATK hackfest, specifically as
something that ideally shouldn't be here (see final note here [3]).

>
> - atk interfaces are only weakly specified (if at all) - no
> information about expected object life-cycles, no information about
> expected signal emissions and order
>
> - the atk interfaces are implemented in a way that forces them to be
> separate from the regular GTK+ apis - these apis were designed a long
> time ago, and use GObject features in weird ways, or use obscure
> GObject features when much more straightforward implementations would
> be possible

In general ATK was obsolete, had a lot of plumbing solutions, and a lot
of inner logic and decisions that the original ATK developers knew, but
was lost when they stopped to work on ATK, as the documentation was also
not enough (or at least centralized).

One of the main reasons of the ATK hackfest was review the features and
interfaces of ATK, to know what we have and what we want to clean and
improve. Something that would require an API change. It also added the
idea of write guidelines, in the sense of, "what an AT expect from us".
Anyway, it is true that most of the conclusions from that hackfests are
pending, and we just made some of those actions.

>
> - there's many layers between the application and the ATs: application
> widgets - gail - atk-bridge - (a11y) bus - libatspi - ATs

For the future, the idea is having ATK as interfaces and not as objects,
so probably one of the layers could be removed. Not sure how to remove
the other layers. For example, the idea of the bridge is being as small
as possible, mostly doing the registration and the communication
(emitting gobject signals as DBUS messages, etc). Something like that
would be required somehow.

Anyway, agree that this is something that is required to be debated.

>
> - there are only very few consumers of this interface, and they are
> not of interest to most desktop users and developers

This is also true for most of the accessible technologies for any other
desktop or developer.

>
>
> What are our options ?
> --------------------------------
>
> Possible remedies include:
>
> - Change approach
>
> Instead of 'special interface for the 1% of users that really need
> it', go for something that is useful for everybody, then add the few
> a11y extras that get you from 'can be used by 80%' to 90%, then 95%,
> ...
>
> - Reduce scope
>
> Look at what concrete ATs actually use, instead of clinging to some
> overly broad inherited 'standard' interface.
> My guess is that this will boil down to mainly text + focus.

Not sure about how far reduce the scope. Totally agree that current text
management needs a improvement (or replacement with IA2).

>
> - Drop layers
>
> Consider implementing D-Bus interfaces in the toolkit itself, cut out
> the bridge and other intermediate layers as far as possible.
>
> - Broaden the audience
>
> Candidates for non-a11y consumers of the interface include:
>  - input methods
>  - scripting
>  - tests
>  - general-purpose voice control
>
> - Integrate
>
> Add a11y features to the desktop shell, instead of focusing solely on
> standalone ATs.
> Examples:
>  - add scanning and other a11y features to the shell OSK

That was always the plan. AFAIK, Caribou itself has scanning and other
a11y features. Ideally it would be good to expose those from the
specific GNOME Shell Caribou view.

>  - voice control in the regular desktop

Not sure about this approach. In the end users requiring voice control
would also require to use specific applications.

>  - touch-friendliness should have some synergy with a11y

Sure. In fact we were talking briefly about gestures and multi-touch on
the hackfest. Unfortunately we didn't have any conclusion, as no one in
the room knew too much about it. We need:
  * Check how gestures/multi-touch are being implemented these days
  * How those are already being used in other platforms for a11y (iPhone
is a really a11y-friendly device, for example [4])

>
>
> Can we turn this into a concrete project ?
> -----------------------------------------------------------
>
> No finished proposal, just some starting points:
>
> - Put concrete goals:
>   'We need a working screen reader that can be turned on and off at will'
>   'No a11y-induced crashes'
>
> - Do research:
>   - What features are expected of a screen reader nowadays ?
>   - Study state-of-the-art offerings on Win32 / OS X (screen readers)
>
> - Investigate if we can have a more limited API that serves the common
> need of screen reading, zoom and screen keyboard: text, labelling
>   relations, focus tracking, change events.
>
> This is not an easy project. There are tough architectural issues
> involved here. Just hiring somebody for a year with the task of 'fix
> accessibility' is almost guaranteed to end in failure.

I totally agree. Although this could sound harsh, IMHO this was one of
the problems with previous approaches. Accessibility was implemented as
an add-on feature, parallel and (almost) independent to the rest of the
technological architecture, when this is a complex thing that should be
view from earlier stages, as a design decision [5].

Finally, thanks for the interesting analysis.

BR
[3] https://bugzilla.gnome.org/show_bug.cgi?id=649559#c1
[4]
http://www.tuaw.com/2010/09/20/blind-user-explains-why-he-loves-the-iphone/
[5] http://opensourcebridge.org/sessions/670

-- 
Alejandro Piñeiro Iglesias