Re: [g-a-devel] RFC: keyboard/mouse usage for accessibility?



Eric Johansson, le lun. 18 mars 2019 14:31:54 -0400, a ecrit:
On 3/18/2019 2:15 PM, Samuel Thibault wrote:
Eric Johansson, le lun. 18 mars 2019 13:44:04 -0400, a ecrit:
I'm a speech recognition user and quite frankly, almost all the
accessibility features people have put in place do not work  with speech
recognition. How can we get our needs injected in the conversation?
By talking about it :)

Thanks :-) where should I be talking about it?

Here is a good start :)
Just make sure to keep gnome-accessibility-devel gnome org in Cc since
that's where at-spi is mostly discussed AFAIK.

I should be able to just say "restart numbering" and have it activate
that "right click, hunt through the menu and click on the item"
action.
Right, this is actually supported in at-spi, actionable widgets are
supposed to have a list of actions with a name for each, that can be
used in such a situation (and could be shown to the user so she knows
she can trigger it). I have also added that to the list.

Is there any capabilities in at-spi that allow a speech recognition
environment to query the application and find out enough context to be
able to generate appropriate grammars? for example, using  a
multipurpose editor. I may want to have different grammars for different
types of data. I need to know what tab has focus and something about
that tab (TBD) so that I can generate the right grammar to operate on
data within that tab.

At-spi provides information of which widget has focus, and then with
at-spi you can inspect the list of actions.

Each cell of the database has a type and a name. For text fields,
saying "Change <name>" Should put me in the cell of that name. But if
the cell is a multi-selection list,  the grammar should be all of the
options for the multi-selection list. If if the cell is a number, I
want to limit the grammar to just numbers.

AFAICT, the type of content is not exposed in at-spi, but that could be
added.

There are a bunch of other types such as email address phone number URL
people and other limited board non-English language elements that could
be spoken. Each of which need their own grammar.

The exact allowed grammer could be passed through at-spi.

One of the problems though with the notion database is that there are no
row names except by convention. Therefore whenever you use a name to
sell, somebody needs to keep track of the role you are on and no
command  should take you off of that row.

I'm not sure to understand.

The last issue is a method of bypassing JavaScript  backed editors. I
cannot dictate into Google Docs, have difficulty dictating into
LinkedIn. In the browser context, only naked text areas seem to work
well with NaturallySpeaking.

That is the kind of example where plugging only through at-spi can fail,
when the application is not actually exposing an at-spi interface, and
thus plugging at the OS level can be a useful option.

Samuel


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]