GNOME.org
 

Re: [orca-list] Copying Math Expressions from Web pages

Hello,
so, it eventually took me more time than I thought, but doesn't matter,
here we go:
https://rastisoftslabs.com/?smd_process_download=1&download_id=359

This is an extended version of the last Chinfusor release. You can set
it up in the same way as normal Chinfusor, see documentation.
The only difference in setup is, that you need to create a file
controller.dat in ~/.config/chinfusor
Then your Chinfusor will be ready to receive commands through this
file.

How to Activate and finish recording?

There is a program called chinfusor_recording_activator. This program
when launched will switch recording to opposite state i.e. from off to
on and wiceversa.
I recommend placing this program to a stable destination, like
/usr/bin. Then you need to create a keyboard shortcut for it.

There are various ways to do this, for example on the latest Ubuntu
mate, you can do the following:
1. Press alt+f1, navigate to System/Settings/Hardware/Keyboard
shortcuts.
2. In the opened window, press Add button.
3. Enter name for the new shortcut and command to execute
chinfusor_recording_activator i.e.
/usr/bin/chinfusor_recording_activator, if you've put it into this
location. By including --accelerated flag in the command, you can
activate accelerated recording (explained later).
4. After confirming the shortcut, navigate to Common shortcuts in the
tree and find your newly created shortcut, with status disabled. To
assign it, press right arrow and space, press your desired key
combination and leave by tabulator.
5. After checking that the shortcut was really assigned in the tree,
you can close the dialog.

Now, when pressing the activation shortcut, Chinfusor will say
"Recording", anything you cross with Orca will be catched. When you
press the shortcut again, Chinfusor will say "Copied", and the
recording will stop, with recorded text appearing in your clipboard.

Accelerated recording


From my point of view, Orca takes a really strange approach for reading

things. Instead of pre-synthesizing text to be spoken, it pushes every
single element into a queue, what causes a whole series of problems,
like broken intonation, too long pauses in speech, and in this case, a
requirement to wait until the last math element is read, otherwise
Chinfusor simply won't get the whole text from Orca.
And because of the mentioned long pauses, this can take a while to do.
I have thus thought out an accelerated recording mode. This mode fools
Orca by claiming that the text was read immediately after receiving it.
Thus Orca gives out another part, which is again claimed to be read,
ant this way it goes on, until there is nothing left. This way, an
equation is copied almost immediately.
The only drawback of this method is, that whatever is received is not
being really read, as it would mess up the output tags. It's like
entering a silence mode.
Thus the best application of this technique is, when you have cursor
one line above formula, you can activate accelerated recording, press
down arrow, and disable recording almost immediately.

Limitations

There are few limitations of recording in this way, which I didn't have
time yet to solve. One of them is, that because the activator is a
console app, it blinks on the screen for a moment, what causes Orca to
read title of the currently active window, which is then catched by
Chinfusor.
Another thing is, that Orca for some reason adds period after every
part of math expression. I don't know why, it just does it. I didn't
know whether it is a problem or not, so I didn't include a filter yet.
Otherwise, I found the copying proces to be quite quick and reliable.

Also, thanks for your interest in this topic, I hope I will be able to
produce betas of MathPresenter soon.
Yes, statistics is very close to data sciences, and artificial
intelligence, infact, the whole human brain is just one big statistical
machine. What you use in traditional statistics and psychology to find
phenomenas in data, is just run in a very complex system, which not
just finds phenomenas on data, but phenomenas on phenomenas, another
phenomenas on those phenomenas and soon to create abstraction in the
system and use it to do something useful. It's fascinating subject and
I definitely recommend looking on it, if you're interested in such
stuff.
I use Sagemath as my CAS, as it unifies various mathematical libraries
such as R, Maxima, SymPy or NumPy in a unified Python interface. Python
is not just a full-fledged programming language, but it is also very
popular in science applications, what means very well developed
libraries for data processing, machine learning and deep learning, like
numPy and TensorFlow. When I take in the fact, that I know Python
already, it was infact a decided thing. :)
SageMath is interestingly more accessible on Linux than Windows, the
Windows version required a trick to work with Jaws at all, and even
then, the console acted quite strangely.
Reading math is another story though. :)
Access8Math has better parsing than Jaws, I just don't understand who
to the heck thought out to place manual pauses between all nodes of an
equation! The reading sounds like a stucking machine, and when I
actually opened the structural navigation and saw, that each elements
gets Item number before it is actually announced, it seemed like some
kind of bad joke.
Bud it worked at least, so it was a good replacement on places, where
Jaws parser got stuck. Jaws has the best math reading style I've ever
seen, it presents formulas in a natural, very practical way, without
any slowing down or useless notes. The whole screenreader is very well
tuned, and the math support is great... when it works.

Best regards

Rastislav
V Nedeľa,  9. august 2020 o 21:12 +0200, Ishe Chinyoka napísal(a):

Hi,


On Sunday,  9 August 2020, at 20:33, Rastislav Kiss via orca-list <
orca-list gnome org> wrote:

Hello,
heh, it seems we are in somewhat similar position, although coming
from
different backgrounds. I'm using Linux as my main operating system
for
about 3 months by now, and I have problems with reading math too.


Sure, it looks like maths accessibility is still a challenge with
most
screen-readers. On Windows, I used to rely on NVDA's Access8Math
addon
to read maths content. But seems it could not copy maths expressions
as
they are said out by speech. 


But if you want to really do it this way, there is an option. I
have
made a project called Chinfusor some time ago:
https://rastisoftslabs.com/2020/07/22/chinfusor-a-universal-solution-for-reading-texts-in-foreign-alphabets-on-linux/

It's a speech module for speech dispatcher, which acts as a middle
layer between clients and speech modules in order to route parts of
text written in various alphabets to specified engines with
specified
configuration.


I am Now downloading it to my machine, seems to be a promising
project
not only for Maths but even some other content that I come across on
the
web where some letters are not voiced out well, I just hear "Chinese
leter, Chinese letter".


But never mind that now, the important thing is, that it's a speech
module, receiving all spoken text from Orca. If you want, I can
make a
special build for you, which will allow recording incoming speech
requests with a keyboard shortcut, and copying the recorded text to
clipboard.
Thus, you would be able to copy an equation like:
Press shortcut,
make Orca speak the equation you want to get,
press shortcut again, the collected text will appear in your
clipboard.


I think this is what I am looking for when dealing with lots of
formulas
on the web. Though translators may be the best as you point out, but
such a simple approach for me for a start would be okay. In fact,
most
of the time what I do is even though I get a formula through  speech,
I
usually look for its equivalence in LaTeX if I want to be sure of how
it
has to be written down. So for a start, this module promises to be a
lifesaver instead of copying what the speech is saying to the
keyboard
by hand like what I am doing right now. Others who may also be
engaged
in technical work may find this module useful for their productivity,
I
would like to believe so.


Although I would personally most likely go with some kind of
translator
to LaTeX or similar format, like:
https://sourceforge.net/projects/xsltml/


Let me check this one out as well. I am interested in compiling as
much
info as I can on how maths can be accessed, or better still how we
can
script this to do it as end users.



Can you programm in JavaScript? If yes, plugins like TamperMonkey
can
make copying MathML notation much more flexible, technically if you
connect it with sheets above, you should be able to get directly a
LaTeX form. I can't confirm quality of the sheets, as I didn't use
them
myself, I just saw some people using them, so they'd be my first
try if
i wanted a functionality like this.


I have some familiarity with JavaScript as it is used in NodeJS
applications. For the browser, it had been a long time since I wrote
it.
But I am willing to learn and devote enough time to find out what
needs
to be done. Thanks for pointing out as well.



I personally find the current Orca MathML interpretation quite
weak. It
not just doesn't contain a structural mathematical navigation, but
even
the basic reading makes quite strange things, such as on this page:
https://openstax.org/books/calculus-volume-2/pages/3-6-numerical-integration


I can confirm that as well, because on one page yesterday, I knew
that a
formula must have a square root in it, but it just said "operator"
and
"end operator". However, I think it does a better job as its maths
support is passable out of the box as compared to Windows
screenreaders
that may need some addon to do just that.



Thus I'm working on my own MathML parser to do this. I originally
thought, that i will create presentation markup parser along with
content markup parser, but considering the fact that even Firefox
doesn't support content markup yet, I'm currently thinking about
another plan, splitting the project to two parts, what would allow
me
to release the thing much sooner than originally expected.

It's not ready yet, but if I finish the tags implementation, I will
need betatesters to check it with various formulas. I want a rock-
solid 
parser, I had quite bad experiences with Jaws on Windows, where
many
formulas simply got the interpreter stuck, draining cpu to 80 or
90%. I
don't want anything similar on Linux, but that means the program
must
be checked in first place. Would you be interested in this?


Sure, I would be interested in testing out this project as it
directly
relates to my work. It offers a promise for a better user experience
on
*nix platforms. 

Also, if yes, I would also like to ask, what area of mathematics is
your most used? For example, i study artificial intelligence and
deep
learning, which is almost completely built on multidimensional
calculus, which could be in theory coverable quite easily, as it
contains rather equations than mathematical statements.


My work is in the social sciences, particularly international
politics
and relations. I mostly deal with quantitative research material,
which
means that my branch  of maths is statistics.  Of recent though, I
had
been trying to dabble into data science as I see it as glorified
statistics with some computer science thrown in to appeal to the
business world, otherwise I see no major difference with traditional
statistics we grew up studying. So in short, I am looking forward to
full access to statistics. Though R (and ESS in Emacs) is perfect for
my
production needs, it is to reading that the issue comes.

So I will be looking forward to any module that can help in maths
access.


Best regards


Thanks for your pointers and these promising projects. This is what
motivates us to learn more about the internals of Linux itself to
give
back to the community that had been helping us stay ahead of our
game.
[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]