[g-a-devel] Extending at-spi2 to support touch input.

[Luke Yelavich <luke yelavich canonical com>]

Hi Folks,
Finally, there is one more point of discussion I would like to start on, and this does go somewhat in hand 
with my at-dpi mail.

As previously stated, the plan is to get Orca support up and running with the Ubuntu mobile/touch release for 
14.04. To bring this to full fruition, at-spi2-core will also need to be extended to process touch input 
events in a similar way to how it processes keyboard events now. Touch events are somewhat similar to mouse 
movement, in that the position of a finger motion needs to be tracked, and the direction needs to be 
calculated to determine the gesture being performed. From what I've seen, multi-touch hardware supports up to 
4 fingers, so up to 4 fingers, and possibly more in the future would need to be tracked.

The other concern is gestures themselves, and whether it would be better for at-spi2 to have a pre-defined 
and extendable list of gestures, and have AT clients deal with gesture events by name, or should at-spi2 pass 
all gesture events through to the AT and let it do the tracking and calculation. The former may be 
appropriate for use cases where the user wants to choose a particular gesture for an Orca command, making it 
easier to choose the desired gesture from a list, instead of actually performing the gesture themselves. The 
latter would allow the user to designate totally custom gestures to commands, such as double tapping a 
particular section of the screen, with a certain number of fingers. The concern then would be providing an 
audible representation of that gesture to the user when they navigate through the commands. Of course, it may 
be decided that touch gestures for Orca are not customisable.

I think X has a touch input API, so we have a way to start implementing touch support right away, and be able 
to test it, and as work goes on to enable at-spi2 for use with other display servers, we can then add touch 
input support as part of that work.

Once again, discussion, thoughts and suggestions welcome. I'll likely make a start on this in the coming 
weeks, along with the Orca configuration work, once we have a consensus on both tasks.

Thanks again for your time and consideration with all these points of discussion.

Thanks

Luke