Re: [orca-list] Contribution for Object Navigation

[Rastislav Kiss <rastislav kish gmail com>]

Hello,
object navigation itself could be an interesting thing imo, as it
currently sometimes happens, that a floating window pops up, and it's
not possible to approach it once you leave it, or at least I didn't
find a way to do so.

As for MathML, here things become slightly more complicated. Problems
could be splitted into three categories:
* Polymorphic behavior. It's true, that most math elements can be
treated as object tree, navigating with the same philosophy, but tables
for example, which are also a part of MathML, and they're used quite a
lot, require a different approach to be used efficiently.
* Polymorphic presentation, various MathML elements require various
presentation strategies, which often depend on their position in tree.
For example, in a fraction, first element should be announced as
numerator, whereas second element as denominator. While it technically
could be possible to do this, it would mean lot of MathML specific code
in infact universal accessibility technique.
* Confort, object navigation is designed for access to otherwise hardly
available areas, such as various labels, non-focusable elements etc.
Its keyboard shortcuts reflect this goal, being composed of three keys
in case of NVDA, with a reasoning, that user won't use them as his /
her primary navigation method, just a supplementary tool. On the other
hand, structural math navigation is often the only way how to
efficiently get familiar with a very complex formulas, and its user's
primary access technique. Using shortcuts consisting of two or three
keys would be not just unnatural, but also somewhat exhausting.

Especially the second category would be very unpleasant if we wanted to
support content markup of MathML, which requires individual processing
of over 100 various math structures. The whole thing would need to be
either preprocessed first for conversion to presentation markup, what
would lead to unnecessary information loss, or we would need to
transform the object tree in a way, that apply tags would be made for
each structure as a separate object and would fake objects under them.
Neither way seems very promising to me...
Or, another option would be to introduce a completely new objects
collection, which would be initialized using the MathML notation.
Polymorphism would be still a problem though.

So, to sum it up, do we need an object navigation in Orca? In my
opinion yes.
Should object navigation handle Math expressions? I would say no.

The current way which I'm working on includes sending MathML notation
over tcp to server, which will process it and either display a Window
with navigation options if requested, or return a string containing the
equation in a form to be spoken.
An advantage of this approach is not just that it can be included in
Orca quite easily, but also people can create multiple types of
MathServers, reflecting warying users needs, such as NVDA has currently
two possible Math providers.
And at the same time, I think I've found out a way to do this even
without Orca, although I didn't checked it yet as I'm still mostly
loading expressions from files to have a greater control over their
testing.

Best regards

Rastislav
V Pondelok,  10. august 2020 o 19:39 +0200, Rynhardt Kruger via orca-
list napísal(a):
> Hi everyone,
>
> With relation to the recent discussion about reading mathematics,
> I've
> wondered if object navigation functionality can be used to read
> mathematical equations in more detail. I'm specifically referring to
> the functionality provided by NVDA and Voiceover, whereby one can
> navigate through the object hierarchy of the UI. Note that as far as
> I
> know NVDA doesn't actually use object navigation for equation
> reading,
> but rather gets MathPlayer to handle it.
> To test whether object navigation could be useful for equations, I
> made a small implementation for Orca over the long weekend, and it
> seems to work reasonably well. My implementation can also be used for
> navigation in general. I also try to simplify the object hierarchy
> somewhat, by removing panels and other elements without interesting
> text, though this might require some tuning still. Also, in the case
> of maths in particular, I'm not sure how to include things like the
> start and end of fractions, super and subscript indicators, and other
> non-symbolic indicators in the hierarchy. These things get read out
> when encountered as part of an expression or sub-expression, but not
> at the leaves of the object tree. (I should note that I use Orca's
> excellent generator functionality to produce the actual announcements
> for each object.)
>
> So my question is, would something like this be useful enough to be
> included in Orca itself? If so, I can make a merge request on Gitlab.
> If not, it might be useful as an extension for Orca, but I'm not
> exactly sure how it could be packaged.
> I could also just upload it to a github repo for further input.
>
> Thanks and regards,
>
> Rynhardt
> _______________________________________________
> orca-list mailing list
> orca-list gnome org
> https://mail.gnome.org/mailman/listinfo/orca-list
> Orca wiki: https://wiki.gnome.org/Projects/Orca
> Orca documentation: https://help.gnome.org/users/orca/stable/
> GNOME Universal Access guide: 
> https://help.gnome.org/users/gnome-help/stable/a11y.html