Re: [orca-list] Contribution for Object Navigation

[Rynhardt Kruger <rynkruger gmail com>]

Hi Rastislav,

I definitely agree with you that a more sophisticated interface is
necessary for effective mathematics exploration. However, for the
moment it seems that object navigation would at least allow one to
"drill down" into an equation in order to have it read out in shorter
chunks.

I also agree that the hierarchical approach is probably not the best
way of presenting mathematics to the blind/VI user, even though it
seems to have received quite a lot of attention in the literature. I
would for instance prefer moving between the numerator and denominator
with up and down arrows, much like one would move between lines on a
page or staves in music score editors. There is also an argument to be
made for aligning the interface for reading maths, with the way one
would write or do maths, but perhaps we're not quite there yet.
Personally I do maths by using something like boiled-down LaTeX.
Something which is simple enough to be easily edited and reviewed with
normal text navigation commands, and yet close enough to LaTeX so that
a sighted person can understand (or at least my lecturers did in
undergrad.)

Regards,

Rynhardt


On Mon, Aug 10, 2020 at 9:04 PM Rastislav Kiss via orca-list
<orca-list gnome org> wrote:
> Hello,
> object navigation itself could be an interesting thing imo, as it
> currently sometimes happens, that a floating window pops up, and it's
> not possible to approach it once you leave it, or at least I didn't
> find a way to do so.
>
> As for MathML, here things become slightly more complicated. Problems
> could be splitted into three categories:
> * Polymorphic behavior. It's true, that most math elements can be
> treated as object tree, navigating with the same philosophy, but tables
> for example, which are also a part of MathML, and they're used quite a
> lot, require a different approach to be used efficiently.
> * Polymorphic presentation, various MathML elements require various
> presentation strategies, which often depend on their position in tree.
> For example, in a fraction, first element should be announced as
> numerator, whereas second element as denominator. While it technically
> could be possible to do this, it would mean lot of MathML specific code
> in infact universal accessibility technique.
> * Confort, object navigation is designed for access to otherwise hardly
> available areas, such as various labels, non-focusable elements etc.
> Its keyboard shortcuts reflect this goal, being composed of three keys
> in case of NVDA, with a reasoning, that user won't use them as his /
> her primary navigation method, just a supplementary tool. On the other
> hand, structural math navigation is often the only way how to
> efficiently get familiar with a very complex formulas, and its user's
> primary access technique. Using shortcuts consisting of two or three
> keys would be not just unnatural, but also somewhat exhausting.
>
> Especially the second category would be very unpleasant if we wanted to
> support content markup of MathML, which requires individual processing
> of over 100 various math structures. The whole thing would need to be
> either preprocessed first for conversion to presentation markup, what
> would lead to unnecessary information loss, or we would need to
> transform the object tree in a way, that apply tags would be made for
> each structure as a separate object and would fake objects under them.
> Neither way seems very promising to me...
> Or, another option would be to introduce a completely new objects
> collection, which would be initialized using the MathML notation.
> Polymorphism would be still a problem though.
>
> So, to sum it up, do we need an object navigation in Orca? In my
> opinion yes.
> Should object navigation handle Math expressions? I would say no.
>
> The current way which I'm working on includes sending MathML notation
> over tcp to server, which will process it and either display a Window
> with navigation options if requested, or return a string containing the
> equation in a form to be spoken.
> An advantage of this approach is not just that it can be included in
> Orca quite easily, but also people can create multiple types of
> MathServers, reflecting warying users needs, such as NVDA has currently
> two possible Math providers.
> And at the same time, I think I've found out a way to do this even
> without Orca, although I didn't checked it yet as I'm still mostly
> loading expressions from files to have a greater control over their
> testing.
>
> Best regards
>
> Rastislav
> V Pondelok,  10. august 2020 o 19:39 +0200, Rynhardt Kruger via orca-
> list napísal(a):
> > Hi everyone,
> >
> > With relation to the recent discussion about reading mathematics,
> > I've
> > wondered if object navigation functionality can be used to read
> > mathematical equations in more detail. I'm specifically referring to
> > the functionality provided by NVDA and Voiceover, whereby one can
> > navigate through the object hierarchy of the UI. Note that as far as
> > I
> > know NVDA doesn't actually use object navigation for equation
> > reading,
> > but rather gets MathPlayer to handle it.
> > To test whether object navigation could be useful for equations, I
> > made a small implementation for Orca over the long weekend, and it
> > seems to work reasonably well. My implementation can also be used for
> > navigation in general. I also try to simplify the object hierarchy
> > somewhat, by removing panels and other elements without interesting
> > text, though this might require some tuning still. Also, in the case
> > of maths in particular, I'm not sure how to include things like the
> > start and end of fractions, super and subscript indicators, and other
> > non-symbolic indicators in the hierarchy. These things get read out
> > when encountered as part of an expression or sub-expression, but not
> > at the leaves of the object tree. (I should note that I use Orca's
> > excellent generator functionality to produce the actual announcements
> > for each object.)
> >
> > So my question is, would something like this be useful enough to be
> > included in Orca itself? If so, I can make a merge request on Gitlab.
> > If not, it might be useful as an extension for Orca, but I'm not
> > exactly sure how it could be packaged.
> > I could also just upload it to a github repo for further input.
> >
> > Thanks and regards,
> >
> > Rynhardt
> > _______________________________________________
> > orca-list mailing list
> > orca-list gnome org
> > https://mail.gnome.org/mailman/listinfo/orca-list
> > Orca wiki: https://wiki.gnome.org/Projects/Orca
> > Orca documentation: https://help.gnome.org/users/orca/stable/
> > GNOME Universal Access guide:
> > https://help.gnome.org/users/gnome-help/stable/a11y.html
>
> _______________________________________________
> orca-list mailing list
> orca-list gnome org
> https://mail.gnome.org/mailman/listinfo/orca-list
> Orca wiki: https://wiki.gnome.org/Projects/Orca
> Orca documentation: https://help.gnome.org/users/orca/stable/
> GNOME Universal Access guide: https://help.gnome.org/users/gnome-help/stable/a11y.html