Re: [orca-list] Contribution for Object Navigation



Hi Rastislav,

I definitely agree with you that a more sophisticated interface is
necessary for effective mathematics exploration. However, for the
moment it seems that object navigation would at least allow one to
"drill down" into an equation in order to have it read out in shorter
chunks.

I also agree that the hierarchical approach is probably not the best
way of presenting mathematics to the blind/VI user, even though it
seems to have received quite a lot of attention in the literature. I
would for instance prefer moving between the numerator and denominator
with up and down arrows, much like one would move between lines on a
page or staves in music score editors. There is also an argument to be
made for aligning the interface for reading maths, with the way one
would write or do maths, but perhaps we're not quite there yet.
Personally I do maths by using something like boiled-down LaTeX.
Something which is simple enough to be easily edited and reviewed with
normal text navigation commands, and yet close enough to LaTeX so that
a sighted person can understand (or at least my lecturers did in
undergrad.)

Regards,

Rynhardt


On Mon, Aug 10, 2020 at 9:04 PM Rastislav Kiss via orca-list
<orca-list gnome org> wrote:

Hello,
object navigation itself could be an interesting thing imo, as it
currently sometimes happens, that a floating window pops up, and it's
not possible to approach it once you leave it, or at least I didn't
find a way to do so.

As for MathML, here things become slightly more complicated. Problems
could be splitted into three categories:
* Polymorphic behavior. It's true, that most math elements can be
treated as object tree, navigating with the same philosophy, but tables
for example, which are also a part of MathML, and they're used quite a
lot, require a different approach to be used efficiently.
* Polymorphic presentation, various MathML elements require various
presentation strategies, which often depend on their position in tree.
For example, in a fraction, first element should be announced as
numerator, whereas second element as denominator. While it technically
could be possible to do this, it would mean lot of MathML specific code
in infact universal accessibility technique.
* Confort, object navigation is designed for access to otherwise hardly
available areas, such as various labels, non-focusable elements etc.
Its keyboard shortcuts reflect this goal, being composed of three keys
in case of NVDA, with a reasoning, that user won't use them as his /
her primary navigation method, just a supplementary tool. On the other
hand, structural math navigation is often the only way how to
efficiently get familiar with a very complex formulas, and its user's
primary access technique. Using shortcuts consisting of two or three
keys would be not just unnatural, but also somewhat exhausting.

Especially the second category would be very unpleasant if we wanted to
support content markup of MathML, which requires individual processing
of over 100 various math structures. The whole thing would need to be
either preprocessed first for conversion to presentation markup, what
would lead to unnecessary information loss, or we would need to
transform the object tree in a way, that apply tags would be made for
each structure as a separate object and would fake objects under them.
Neither way seems very promising to me...
Or, another option would be to introduce a completely new objects
collection, which would be initialized using the MathML notation.
Polymorphism would be still a problem though.

So, to sum it up, do we need an object navigation in Orca? In my
opinion yes.
Should object navigation handle Math expressions? I would say no.

The current way which I'm working on includes sending MathML notation
over tcp to server, which will process it and either display a Window
with navigation options if requested, or return a string containing the
equation in a form to be spoken.
An advantage of this approach is not just that it can be included in
Orca quite easily, but also people can create multiple types of
MathServers, reflecting warying users needs, such as NVDA has currently
two possible Math providers.
And at the same time, I think I've found out a way to do this even
without Orca, although I didn't checked it yet as I'm still mostly
loading expressions from files to have a greater control over their
testing.

Best regards

Rastislav
V Pondelok,  10. august 2020 o 19:39 +0200, Rynhardt Kruger via orca-
list napĂ­sal(a):
Hi everyone,

With relation to the recent discussion about reading mathematics,
I've
wondered if object navigation functionality can be used to read
mathematical equations in more detail. I'm specifically referring to
the functionality provided by NVDA and Voiceover, whereby one can
navigate through the object hierarchy of the UI. Note that as far as
I
know NVDA doesn't actually use object navigation for equation
reading,
but rather gets MathPlayer to handle it.
To test whether object navigation could be useful for equations, I
made a small implementation for Orca over the long weekend, and it
seems to work reasonably well. My implementation can also be used for
navigation in general. I also try to simplify the object hierarchy
somewhat, by removing panels and other elements without interesting
text, though this might require some tuning still. Also, in the case
of maths in particular, I'm not sure how to include things like the
start and end of fractions, super and subscript indicators, and other
non-symbolic indicators in the hierarchy. These things get read out
when encountered as part of an expression or sub-expression, but not
at the leaves of the object tree. (I should note that I use Orca's
excellent generator functionality to produce the actual announcements
for each object.)

So my question is, would something like this be useful enough to be
included in Orca itself? If so, I can make a merge request on Gitlab.
If not, it might be useful as an extension for Orca, but I'm not
exactly sure how it could be packaged.
I could also just upload it to a github repo for further input.

Thanks and regards,

Rynhardt
_______________________________________________
orca-list mailing list
orca-list gnome org
https://mail.gnome.org/mailman/listinfo/orca-list
Orca wiki: https://wiki.gnome.org/Projects/Orca
Orca documentation: https://help.gnome.org/users/orca/stable/
GNOME Universal Access guide:
https://help.gnome.org/users/gnome-help/stable/a11y.html

_______________________________________________
orca-list mailing list
orca-list gnome org
https://mail.gnome.org/mailman/listinfo/orca-list
Orca wiki: https://wiki.gnome.org/Projects/Orca
Orca documentation: https://help.gnome.org/users/orca/stable/
GNOME Universal Access guide: https://help.gnome.org/users/gnome-help/stable/a11y.html


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]