Re: RFC: keyboard/mouse usage for accessibility?

[Eric Johansson <esj eggo org>]

On 3/18/2019 2:15 PM, Samuel Thibault wrote:
> Eric Johansson, le lun. 18 mars 2019 13:44:04 -0400, a ecrit:
> > I'm a speech recognition user and quite frankly, almost all the
> > accessibility features people have put in place do not work  with speech
> > recognition. How can we get our needs injected in the conversation?
> By talking about it :)

Thanks :-) where should I be talking about it?

> I should be able to just say "restart numbering" and have it activate
> > that "right click, hunt through the menu and click on the item"
> > action.
> Right, this is actually supported in at-spi, actionable widgets are
> supposed to have a list of actions with a name for each, that can be
> used in such a situation (and could be shown to the user so she knows
> she can trigger it). I have also added that to the list.

Is there any capabilities in at-spi that allow a speech recognition
environment to query the application and find out enough context to be
able to generate appropriate grammars? for example, using  a
multipurpose editor. I may want to have different grammars for different
types of data. I need to know what tab has focus and something about
that tab (TBD) so that I can generate the right grammar to operate on
data within that tab.

Oh, just thought of a good example. Are you familiar with the
application notion? It's like  Evernote  on steroids. They have a
interesting take on a database. Each cell of the database has a type and
a name. For text fields, saying "Change <name>" Should put me in the
cell of that name. But if the cell is a multi-selection list,  the
grammar should be all of the options for the multi-selection list. If if
the cell is a number, I want to limit the grammar to just numbers.

There are a bunch of other types such as email address phone number URL
people and other limited board non-English language elements that could
be spoken. Each of which need their own grammar.

One of the problems though with the notion database is that there are no
row names except by convention. Therefore whenever you use a name to
sell, somebody needs to keep track of the role you are on and no
command  should take you off of that row.

The last issue is a method of bypassing JavaScript  backed editors. I
cannot dictate into Google Docs, have difficulty dictating into
LinkedIn. In the browser context, only naked text areas seem to work
well with NaturallySpeaking.