Re: Macro recorder using AT-SPI

[Brian Cameron <Brian Cameron Sun COM>]

Tessa:

>I'm trying to write a macro recorder using the accessibility interface,
>AT-SPI.  A macro recorder requires two things from the underlying
>platform:
>
>(1) tracking what the user has done: which buttons were clicked, what
>text was typed, etc.
>
>(2) programmatically invoking user actions, such as clicking on buttons
>and typing in text
>
>I've run the demo programs that come with at-spi in the test/ directory,
>and used the at-poke program to introspect into running programs such as
>the gnome-panel and galeon.  I've also read up on the AT-SPI API to see
>what can be done with the current interface.
>
>Based on what I've read, #2 is easy to do; at-poke for example is able
>to push buttons and invoke menu items programmatically.  I'll be able to
>read the at-poke source and figure out how it's done.
>
>However, I'm having trouble figuring out how to do #1.  I've written
>test programs that register global listeners on the a11y registry,
>listening for various events.  I've tried listeners on:

It sounds like you are listening to various GTK events rather than ATK
events.  You listen to ATK events through the functions 
SPI_createAccessibleEventListener, SPI_registerGlobalEventListener,
etc.  Refer to at-spi/cspi/spi_registry.c for more information about how
to use these functions.

Refer to gnopernicus code in srlow/libsrlow/SRLow.c for an example of
these functions being used.  Refer to the SRL_EVENT_NAME* variables
to see examples of ATK signal strings.

>A) "mouse:button" -- but the event source is always the desktop, even
>when I click in an application, so I can't tell what button was invoked
>
>B) "Gtk:GtkWidget:button-press-event" --- here I get about 25 event
>notifications each time I push a button (e.g., the foot menu in
>gnome-panel).  I only pushed the button once; why do I get so many event
>notifications?  Anyway this only seems to work for certain buttons.  If
>I bring up the "Search for Files" tool, none of the buttons in that
>dialog box generate event notifications with this listener.
>
>C) "Gtk:GtkButton:clicked" --- I tried setting a listener specifically
>for GtkButtons.  Having to set a listener for every type of
>Gtk widget would be suboptimal, but I'll do it if that's what it takes.
>Even before clicking on anything, I get hundreds of event notifications
>of the "clicked" event on what appear to be the buttons in my WindowList
>applet.  I didn't click them, but this event is firing anyway.  That
>won't work for a macro recorder.
>
>Ideally I'd like to set a listener for whenever the user invokes a
>widget, whether it be clicking on a checkbox, pressing a button, or
>selecting a menu item.
>
>Can someone tell me how to do this with AT-SPI?  (I'm using the Python
>ORBit bindings to speak to the Accessibility interface via CORBA, but
>examples using the AT-SPI C API would be fine, as it's straightforward
>to translate between the two.)

Hope this is helpful!

Brian