Module proposal: LSR

["Peter Parente" <parente gmail com>]

The following is a proposal about including LSR in GNOME desktop
releases. It is a proposal about offering users and developers a
different option in terms of an extensible assistive technology. It is
in no way an attempt to supplant Orca as the default GNOME screen
reader and magnifier. Nor is it intended to undermine the progress the
community has made in terms of accessibility. We'd like to discuss our
work openly and honestly with the GNOME community. I believe this is
possible without inadvertently starting World War III. :)

In this post, I plan to tackle the boilerplate "new module
requirements" up front, and then ease into the more critical questions
near the end. In my comparison of Orca and LSR, I will attempt to
stick to familiar ground: technical differences. I'd like to leave
questions and comments about the user experience to end-users. I
apologize up front for any inaccuracies, and invite corrections.

== Purpose

The Linux Screen Reader (http://live.gnome.org/LSR) project is an open
source effort to develop an extensible assistive technology (AT) for
the GNOME desktop environment. The goal of the project is to create a
reusable development platform for building alternative and
supplemental user interfaces in support of people with diverse
disabilities.

The primary use of the LSR platform is to give people with visual
impairments access to the GNOME desktop and its business applications
(e.g. Firefox, OpenOffice, Eclipse) using speech, Braille, and screen
magnification. The extensions packaged with the LSR core are intended
to meet this end. However, LSR's rich support for extensions can be
used for a variety of other purposes, such as:

   * Supporting novel input and output methods, such as joysticks,
web cams, game pads, audio icons, and environmental sounds.
   * Creating interfaces for other users, such as people with
mobility impairments, people with cognitive impairments, children
learning to use a screen reader, and so forth.
   * Supplementing the graphical GNOME desktop with useful audio
feedback for sighted users, such as text-to-speech reporting of events
in a monitored window.

== Target

LSR is being proposed for the desktop module set.
Dependencies

LSR relies on modules already included in GNOME releases. One possible
new dependency is desktop-file-utils from freedesktop.org
(http://www.freedesktop.org/wiki/Software/desktop-file-utils). We are
uncertain if this package is already included with GNOME. If adding it
is a problem, we can easily stop depending on it and use a manual
method to install of our lsr.desktop file instead.

== Resource usage

LSR is already hosted in GNOME's subversion repository. More than five
tarball releases have been posted on GNOME's FTP server over the past
year. Future releases will follow suit. We are currently tracking LSR
bugs in a GNOME bugzilla module. We make extensive use of irc.gimp.net
for developer discussions in #lsr.
Adoption

Ubuntu has been packaging LSR in its universe repository since Edgy.
The SpeakUp Modified accessibility repository
(http://speakupmodified.org) has been packaging LSR for Fedora Core
since version 5. OpenSuse has recently added LSR packages.
GNOME-ness and community
Internationalization

All human-readable strings are marked for translation. Our autotools
process is configured to build and install translations for strings in
the code. Eight translations have been started by the GNOME i18n team
to date.
Documentation

We have created extensive documentation about the LSR project on the
Live! wiki and GNOME website including:

   * A getting started guide for users
(http://live.gnome.org/LSR/GettingStarted)
   * The beginnings of a user guide to be turned into Docbook
documentation viewable with Yelp (http://live.gnome.org/LSR/UserGuide)
   * A tutorial for creating LSR script extensions
(http://live.gnome.org/LSR/ScriptDevelopers/GaimPerkTutorial)
   * API documentation generated using epydoc
(http://www.gnome.org/~parente/lsr/epydoc/index.html)
   * Screen reader speech UI specification
(http://www.gnome.org/~parente/lsr/ui/index.html)
   * LSR architecture specification
(http://www.gnome.org/~parente/lsr/workbook/index.html)
   * Screencasts and presentations (http://live.gnome.org/LSR#pandp)

== Look and feel

The LSR settings dialog and other GUI dialogs use gtk to maintain
consistent look and feel with the rest of the GNOME desktop. LSR
interfaces with gnome-speech to support text-to-speech synthesis (in
addition to other APIs).
Preferences

The front-end for configuring preferences in LSR is abstracted from
the back-end used to serialize data. At present, the front-end allows
configuration through the LSR settings dialog and the command line.
The back-end persists the data in Python pickle files in the user's
home directory. We would like to add a gconf back-end to LSR to better
integrate with the GNOME desktop. Changing our working code simply
hasn't been top priority.

== Accessibility

All dialogs in LSR are accessible. For instance, LSR can read its own
interface to the user to a user with a visual impairment. Other
assistive technologies such as on-screen keyboards should be able to
work with LSR dialogs too.

== Key Questions

= How do Orca and LSR differ technically?

The following are some major differences in the designs of LSR and Orca.

= Extensions

Orca has the concept of scripts which may be keyed to a particular
application or toolkit. The mapping from script to application is
one-to-one, though a script may inherit methods and attributes from
other scripts. For instance, most Orca scripts for applications derive
from a default script providing basic screen reader functionality with
speech, Braille, and magnification. When a specific script isn't
defined for an application, the default script is used. New scripts
are placed in the site-packages/orca/scripts Python package, and the
mapping from script to application name is defined in the settings.py
file or inferred automatically from the name of the script.

Orca also supports the addition of speech factories supporting speech
APIs such as gnome-speech and Speech Dispatcher. Modules containing
these factories are placed in the site-packages/orca directory, and
should be configured for use by editing the settings.py file or chosen
using the Orca preferences dialog.

The Orca default script; the application scripts; the interfaces for
speech, BrlTTY, and GNOME magnifier; and the Orca settings dialog
constitute the Orca user interface.

LSR has four extension types:

   * scripts: Respond to application and user events by producing output
   * devices: Enable input and output with available hardware
   * choosers: Support lengthy dialogue with the user
   * monitors: Display the flow of events to aid developers

Extensions are packaged into profiles which define a user experience.
Both the profiles and extensions are managed by a registrar which the
user can access through the command line and through the LSR settings
dialog (an extension itself). The registrar can also be controlled
programmatically by extensions which may choose to load and unload
other extensions at runtime. Extensions may live anywhere on disk, and
may be registered for system wide use by a super user or local use by
any user.

A key feature of LSR is that there is no default user interface, only
what a profile constructs from the available extensions. For example,
the initial configuration of the "user" profile consists of scripts
that respond to accessibility events using speech and Braille, devices
supporting speech output and BrlTTY, and choosers for the settings
dialog and the scripter loader/unloader dialog. Together, these
extensions create a screen reader experience.

Other profiles, however, can disregard some or all of our screen
reader extensions, and reuse the LSR core for other purposes. For
instance, we've constructed a "reminder" profile for people with
aphasia containing a speech output device and a script that parses
text in a user selected text area containing a user's daily todo list.
Running LSR under this profile provides audible reminders of upcoming
items in the list, which the user may continue to manage in their
favorite editor as plain text. The script defines settings such as how
far in advance reminder should be given, how reminders should be
announced, what should happen if an item is overdue, and so on.
(http://live.gnome.org/LSR/Extensions)

For more on LSR's extensibility, see
http://live.gnome.org/LSR/FrequentlyAskedQuestions#extensible

= Customization

Orca uses a settings.py file to hold its set of global settings. The
available settings are pre-determined at design time, but their values
can be changed by editing this file or using the Orca settings dialog.

LSR supports settings per extension. An extension author may define a
new setting object with a name, type, description, and other critical
values (e.g. min and max). The settings dialog may then use this
information to automatically generate an accessible interface for
configuring the option. Any setting may be marked for persistence.

For instance, our screen reader script for Gaim defines a boolean
option named "Read background messages?" This option appears in the
LSR settings dialog as an accessible checkbox. As another example, our
IBM speech device extension has an option for baseline pitch. Since
this setting is an integer numeric in a finite range, it appears as a
slider in the settings dialog for the user to configure.

The existence of the extension system provides customization in a
different sense. Functionality may be added, removed, or changed by
adding, removing, or exchanging extensions. If a particular user needs
more descriptive labels on the toolbar buttons in Firefox, for
instance, one might add a new script to his profile providing
additional information when the buttons are read by the other scripts
in LSR. This solution would require no changes to the existing Firefox
script, and could be made available to just this one user.

= Scripting Environment and Adapters

Orca scripts consist of a mixture of calls directly to AT-SPI methods,
helper functions to perform common tasks (e.g. Braille output), and
calls to a base class script to perform default actions.

LSR provides a managed scripting environment with its own API capable
of, but not limited to, the following:

   * querying and manipulating accessible objects on the desktop
   * managing other extensions
   * registering global hotkeys
   * binding actions to arbitrary input device gestures
   * sending and styling output for a device
   * modifying settings with notifications for setting observers
   * modifying event flow
   * reloading scripts on the fly
   * executing code in other scripts without naming them explicitly
(weak dependencies)
   * registering event handlers for signals from the focused control
only, the foreground application, a background application, or any
combination of these

A key characteristic of LSR scripts is that they are devoid AT-SPI
code. Scripts call methods in the LSR scripting API only. The
scripting API, in turn, calls methods on adapter objects wrapping the
platform accessibility API. Which adapter is used to fulfill a request
depends on two factors:

  1. If the adapter can handle the type of object being queried
  2. If the adapter can handle the platform accessibility API in use

The first requirement is checked by assertions defined in each adapter
class. For instance, our TableInfoAdapter class indicates that it will
only work with AT-SPI objects that provide a table interface.
Likewise, our DefaultInfoAdapter class states that it will work with
any AT-SPI object not handled by a more specialized adapter.

The second requirement is determined at runtime by dynamically loading
adapters targeted at the particular accessibility API on the system.
At present, we only have adapters for AT-SPI. But additional adapters
could be defined for accessibility APIs on other platforms (e.g.
IAccessible2 on Windows) or future accessibility APIs not yet defined.

The key benefit of this design is that the LSR scripts are isolated
from changes to the accessibility API. If a new API appears or LSR is
ported to another platform, all scripts will remain largely intact.

= GNOME doesn't ship two of any kind of software. Why should LSR be a
special case?

GNOME packages both Dasher (http://svn.gnome.org/viewcvs/dasher/) and
GOK (http://svn.gnome.org/viewcvs/gok/), two applications termed
on-screen keyboards. These projects appear dissimilar even to
non-users because of the radical differences in their visual UIs. The
differences in interaction styles between Orca and LSR are not as
apparent to the casual observer because most output is auditory or
Braille, and input is done via a keyboard. Nevertheless, we are
certain a screen reader user could report such differences.

= So how do LSR and Orca differ in terms of the user experience?

The technical differences mentioned above lead to differences in the
screen reader user interfaces of Orca and LSR. For instance, the
settings dialog in LSR is more dynamic because script writers can
define options on a per script basis. The available settings will
change based on what extensions the user has loaded. The ability to
layer multiple scripts on an application can also lead to different
experiences. For example, a LSR user might use the script chooser
dialog to load a script that reads Python source code more
intelligently in the active text editor program. Which text editor is
in use does not matter, only that there is some text area in which the
user is editing Python code.

A more drastic difference will be seen if a user is operating under a
profile that loads extensions other than the ones created for screen
reading and magnification. Consider a user running under our example
"reading" profile intended to aid people with reading disabilities.
The scripts in this profile defines a hotkey which the user may press
to see a picture related to the word selected in any application. The
script pulls relevant pictures from Flickr and displays them in a
chooser extension. The chooser has buttons for scrolling forward and
backward through other similar pictures.
(http://live.gnome.org/LSR/Extensions)

Beyond mechanisms for logging, LSR also has a profile with additional
tools suitable for developers looking to check the accessibility of
their applications using a screen reader. Three monitor extensions
show raw AT-SPI events, how the events are processed by scripts in
LSR, and what output the scripts send to devices as a result. A
developer script also defines additional hotkeys for determining what
scripts are active, reloading scripts dynamically at runtime, muting
output temporarily, and so on.

Concerning the differences in the day-to-day experience of using LSR
and Orca, users of both ATs are the most qualified to comment. I would
invite them to do so, if their comments would have any impact on the
acceptance or rejection of this proposal. Please speak up!

In the meantime, we can state some of our LSR UI philosophies which
have a noticeable effect on the screen reader interface:

   * Our default key bindings are based on kinesthetics rather than
language-specific mnemonics. The key bindings are designed to remain
in the same position and make sense in relation to one another no
matter what keyboard or language is in use.
   * The default key bindings are not designed to mimic Windows
screen readers. Again, we're all about position, not learned
mnemonics.
   * The detail provided by the keyboard commands change with the
context, but the underlying meaning stays the same. For example, our
base "Where am I?" query is customized for reporting document context
in Firefox (e.g. under the fifth heading, 50% through the page) versus
standard GUI context (e.g. button in a panel in a frame)
   * The keyboard functions cycle through additional details when
pressed repeatedly. For instance, our "read word" key binding says the
word at the pointer on first press. If pressed again without an
intervening command, it spell that word. If pressed a third time
without intervention, it spells the word phonetically. A fourth press
cycles back to the first function. This design allows us to group
related functionality and avoid an explosion of keyboard bindings.
   * The capabilities of the available audio device are used to
distinguish and aid selection of certain types of information. For
instance, available voices are put to use distinguish content on the
screen (e.g. menu item names), from context information (e.g. window
title), from index information (e.g. item 1 of 10), from LSR error
messages (e.g. failed to get information), from foreground/background
information (e.g. voice to the left is from a background app, voice
dead ahead is at the focus), etc.. Future development will enable
users to define their own mappings from semantic meaning to output
style, or even make everything sound the same if desired.

= Why does LSR have to be part of GNOME? Why not a package in every distro?

Like other applications, an assistive technology is not a one size
fits all solution. Every user has slightly different needs and
preferences, and might prefer one AT over another for certain tasks.
But unlike installing a new text editor, getting a different AT up and
running can be quite difficult without explicit desktop support:

   * Run your current AT
   * Fetch the new AT
   * Quit your current AT and run the new one to ensure it's working
properly (problematic if new AT install went awry)
   * Set the new AT to run after login (difficult today in GNOME)
   * Log out
   * Log in (hopefully the session is configured properly so the new AT starts)
     Having the AT as part of GNOME would make this process easier.
Switching among the available ATs would be tested and debugged before
release, and the burden would not fall on each user or distro.

= Isn't the name "Linux" Screen Reader a misnomer?

You bet. LSR will run on any AT-SPI enabled platform. Moreover, some
wily programmer may decide to port it to a platform in the future.
Calling it Linux Screen Reader doesn't do anyone justice.

If accepted as a GNOME module, we plan on remedying this problem by
changing the project name. This should have minimal impact on the code
base as the name is isolated in all but a few cases. A name along the
lines of Open Screen Reader (open source, open extensibility, open
community) is appealing, but still pigeonholes LSR as a screen reader,
rather than a platform of sorts. We invite better suggestions.

= Won't we need to do more work to make choosing a preferred AT easier
or possible?

We have already done most of the work by creating a new Accessibility
panel for the GNOME Preferred Applications dialog. In this panel, a
user may choose his or her preferred assistive technologies based on
function. The necessary patches against GNOME 2.17 have already been
reviewed and commented on by the proper module maintainers. Some
members of the GNOME, Ubuntu, and Fedora communities have also
contributed. Our patches need to be updated for GNOME 2.19 and
committed. (http://live.gnome.org/GAP/ScratchPad/PreferredApplications)