Re: premature announcement: speech bridge

["Eric S. Johansson" <esj eggo org>]

On 06/22/2015 12:17 PM, Nolan Darilek wrote:
> Hey, this is cool! My GF uses Dragon Naturally Speaking, and has been
> vocal with me that its lack of equivalent under Linux is the only reason
> she still uses Windows.
>
> I recognize it's premature, but mind sharing a few more details? From
> which Windows voice dictation software does it bridge? Also, to what
> extent does it bridge? In listening to my GF's Naturally Speaking use,
> she mainly sticks to dictation plus assorted editing commands ("scratch
> that," "correct X to Y," "new paragraph," etc.) Are those types of
> editing commands supported?
https://docs.google.com/drawings/d/16ai35Qi8eAUNnlrD3CnkGgwa5FS-7gv_itaZ1U6HxXQ/edit?usp=sharing
https://docs.google.com/drawings/d/1M-TzfRaSaAhFXQk1OmcmHNOaW31_7W_7q0bf8CAJqSw/edit?usp=sharing

here are a couple of pieces I created try to explain the concept of 
accessibility bridge.  may or may not clarify  The concept for you.

In a nutshell,  the simple version here captures keysyms and other 
information, converts them to keysyms that can be recognized by the 
Linix emitter side  then via RPC, pushes them to Linux_emitter which in 
turn converts them to linux keysym  sequences and pushes them into the 
input queue.

The next stage will be integrating natlink  into the RPC channel 
allowing you to build a custom grammar, recognized on the Windows side 
but run on the Linux side.

The third stage is figuring out how to make context changes in linux  
drive grammar changes in NaturallySpeaking.

Fifth stage would be correcting recognition errors. I personally like 
the idea of using the recognition history to drive corrections. This may 
only be possible if Nuance helps.

 sixth stage would be to make the partial cursor display work in linux

 sometime after, we really should look at how KVM forwards USB devices 
to make it more reliable/predictable and automated start up/shutdown to 
make windows headless.

> Thanks, will definitely keep an eye on this project. If it is at all
> something she might use, I might look into offering development help if
> there's any way to do so exclusively under Windows (I.e. bridge the
> dictation component to a testing Linux-side adapter to enhance command
> mappings.)

that would be great!