Re: HFOSS Visual Audio


It's nice to see thoughts & proposals around hearing impairments! Regarding your proposal - either as part of an updated proposal, or as part of the first work you proposed to do, I would like to see: (1) a review of the state of the art in other desktops (Mac, Windows, KDE) for visual events, and (2) discussions/connections with places like the TRACE Center & the Nation Center on Accessible Media (NCAM) at WGBH TV around their research in this area. Not that we should simply do as others do, but we should be informed by what others are doing and thinking in this space before we implement something.

On the technical side, in addition to the issues you cite around UNIX audio sub-systems, there is also the question of video functionality that would enable some of the user interface you suggest (e.g. screen dimming). Technologies like Compiz might be a lovely way to do something like this, but then we run into the fact that not every desktop has Compiz (and if you use some other technology, will it be compatible with a Compiz desktop). I think this facet should also be explored - either in preparation to the proposal submission or as work to be funded under the project -> at least to the level of a survey and recommendations.


Peter Korn
Accessibility Architect & Principal Engineer,
Sun Microsystems, Inc.

This is my proposal for visual audio events as a project for the
upcoming HFOSS Summer Program.  We'll call it "Visual Audio" for now but
probably we should come up with a formal title at some point.

I would appreciate comments before we submit it formally to HFOSS.

The Situation:  While open source has done much to reach out to people
with accessibility needs, one group that hasn't had much attention is
Deaf or hearing-impaired users.  In particular, sound events, such as
incoming instant messages, IRC chat beeps, etc. are not heard.


The Proposal:
Other operating systems have come up with solutions such as creating a
screen dimming flickers whenever an incoming message appears.

GNOME/X has some rather rudimentary functions that currently exist.  There is a system alert which flashes the screen and there is of course, the notification daemon.  Neither option works very well as it is not easily configurable.

I'd like to see us implement a similar feature for the GNOME Desktop
Environment, but take it a few steps further.

1)  Allow users to determine what applications send visual events.  If I
was listening to music, I wouldn't want visual effects happening
2)  Allow for customization of visual event effects. This is important,
because like myself, 10% of the Deaf population also lives with Usher
Syndrome (visual impairment.  I think the best approach is to create a
plugin type environment where the general community can contribute by
creating unique effects.  Examples would be:
--Screen dimming flicker
--Hard screen flicker
--Running lights around the border of the monitor
--Graphic popup in designated area of screen  (for me, I miss having
events pop up in the middle of my screen like on Windows.)
--Animated events, such as a snowball splat.  Sounds crazy, but its a
fun approach.

This proposal benefits not only Deaf users, but also the general
population of users who do not have speakers nor wish to enable their
speakers (e.g. in the workplace.)  So it has broad appeal.  Furthermore,
the "eye candy" effect of this application would help to further propel
adoption and mass creation of visual effect plugins.  As such, the
plugin environment should be easy for people to work with.

There are some technical considerations we should also cover.  For
example, Alsa vs. Pulse Audio.  Metacity vs. Compiz.  This app should be
able to function regardless of specific environment factors.

Possibly we could consider tying this into the notification daemon, thus reducing the work that the HFOSS interns have to do with building something from scratch.  There is of course one drawback in that not all applications use the notification pop-up bubbles.  How to work around this is another question to be dealt with.

Feedback on this topic is greatly appreciated.

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]