Re: Help needed: About gnome architecture - auditory desktop

[Eduardo Trapani <etrapani unesco org uy>]

Hi Bill,

> I recommend that you look at the Gnome Accessibility Project pages
> (http://developer.gnome.org/projects/gap) and review the ATK API.

Ok, I am doing it right now.  I would hold the rest of the message until
I have read enough but I can't wait.  It was great to receive such a
prompt response.

> There is work already underway that, in effect, creates ATK "peers"
> for desktop objects.  The textual and other information provided by
> these ATK implementations is intended for use by alternative interface
> presentations, such as auditory interfaces.

I would then just have to deal with the ATK peers and from them gateway
the information to my interface.

> > b) send events to widgets so that I can gateway the input from the
> > auditory desktop to the gnome desktop.

> GTK widgets are being patched to better support keybindings.  Vision
> impaired users should be able to access all of the GTK widget actions
> from the keyboard, and by using ATK an auditory interface can be
> notified of the user's actions.

Mmm, I am not sure about this.  Let's say the auditory interface says
"Ok button" and the user speaks to the microphone "go ahead".  How would
then the auditory interface "click" the button?  Maybe it is not voice,
but there could be a shortcut key to click on the last heard button. 
Would that be covered?

> Perhaps you could describe in more detail the particular applications
> and "use cases" which you would like to support, then we could discuss
> how existing and planned Gnome accessibility interfaces could best be
> used.

Well, this is just an idea.  I have been reading a lot about
accessibility projects (mercator/ultrasonix, etc.), seen patches to
xlib, and what I liked most is the notion of a roomed auditory
interface.  Basically you have rooms where things are located.  Things
are applications, documents or doors.  That is what concerns the
desktop.

For running applications then you could have a menu shelf or ... I don't
know, I have not thought about that part yet and still there area some
documents on auditory interfaces waiting to be read.

My idea is to gather as much information as I can from the widgets and
their relationships and then build an object model that the auditory
desktop can use.

(user) -- [auditory desktop]
                  |
                  |
                 ----------
                (translator)
                 ----------
                         |
                         |
(application) -- [   gnome desktop  ]
                 [gnome applications]

This would be the situation.  The auditory desktop only sees/uses the
objects from the translator.  This isolates the auditory desktop from
gnome, kde or whatever is on the other side.

The translator builds its object model (if there is already a suitable
one it will use that one directly) from the graphical one.  Gnome seems
to be accessible directly from the auditory interface, but I would
rather have the translator, even as a very thin direct-mapping layer,
just for the sake of portability.  Besides there migh be some small
adjustments and certainly the relationship between graphical and
auditory widgets will not be one to one.

The goal is that at least well behaved gnome applications could be
transparently routed to the auditory desktop.

> If you truly want to present a different metaphor, then perhaps GTK+
> is not the right tool, you might consider other ways of providing
> alternate interfaces to applications.

You might be right, but applications do not know they are going to be
accessible and to translate one metaphor to the other I need to know the
logical structure of the graphical one..

I have thought of many ways to do it, but you always have to "cut in"
somewhere and so far Gnome seems to provide the most comfortable place
to sit and have access to everything.  If I were to, for example, create
a pseudo X server then I would be losing a lot of information about the
widgets.  (I think that in this case some "common sense" logic would
help try to guess the logical structure, but it would be really
complicated.)

Any other ideas on how to do it?

Eduardo.