[g-a-devel] Making Wayland accessible

[Matthias Clasen <matthias clasen gmail com>]

As part of the ongoing GNOME Wayland porting effort, the GNOME accessibility (a11y) team has been looking into what it would take to make our existing a11y tools (ATs) and infrastructure work under Wayland.

Most a11y features will probably end up being implemented in the compositor:
 
- input tweaks like slow keys or bounce keys or hover-to-click naturally fit in the event dispatching in the compositor

- display changes like zoom or color adjustments are already handled in gnome-shell under X

- the text protocol should be sufficient to make on-screen keyboards and similar tools  work

The remaining AT of concern is orca, our screen reader, which does not easily fit into the compositor. Here are some examples of the kinds of things orca needs to do to support its users:

- Identify the surface that is currently under the pointer (and the corresponding application that is registered as an accessible client)

- Warp the pointer to a certain position (see https://bugzilla.gnome.org/show_bug.cgi?id=709999 for a description of how this is used)

- Filter out certain key events and use them for navigation purposes. This is currently done by capturing key events client-side and sending them out again via at-spi, which will probably continue to work, even if it is awkward and toolkit authors would really like to get rid of it

All of these features violate the careful separation between clients that Wayland maintains, so that probably calls for some privileged interface for ATs. I would appreciate feedback and discussion on this.

Has anybody else thought about these problems already ?

Are there better ways to do these things under Wayland ?

Matthias