Orca Summary of dogtail/test talk at Boston GNOME Summit (Saturday afternoon)

[David Malcolm <dmalcolm redhat com>]

[cross-posting to Dogtail, LDTP and Orca development lists]

We had an impromptu session on Dogtail on the Saturday afternoon of the
GNOME Summit.

There was a broad range of knowledge of the accessibility stack within
the participants, from "none" to "implemented large sections of it", so
we spent some time giving an overview of how it works, and how you can
hook into it for the purposes of testing.

I then gave an idea of how dogtail is structured, along with a
comparison to how LDTP and Orca work (as I understand them; please
correct me if I've got this wrong):

+--------------------------------+       +------+
|         test suites            |       | Orca |
+--------------------------------+       +------+
        |                 |                 |  (via pyorbit)
+---------------+   +------------+          |
|dogtail scripts|   |LDTP scripts|          |
+---------------+   +------------+          |
        |                 |                 |
+---------------+   +------------+          |
|    dogtail    |   |    LDTP py |          |
+---------------+   +------------+          |
        |                 |                 |
+---------------+   +------------+          |
|     pyspi     |   |   LDTP .c  |          |
+---------------+   +------------+          |
        | (via Pyrex)     | (handwritten)   |
+--------------------------------+          |    +-----------------+ 
|              cspi              |          |    | apps under test |
+--------------------------------+          |    +-----------------+ 
                |                           |             |
+------------------------------------------------+        |       
|          AT-SPI idl                            |        |
+------------------------------------------------+        |
                |                                         |
                +---------------------- CORBA ------------+

I then demoed the Dogtail script recorder, including the keystroke
recording functionality (which people seemed impressed by).  (BTW, turns
out that syntax highlighting is broken in HEAD - I've submitted a fix
for this to bugzilla this morning)

We brainstormed some ideas for testing that a11y is working properly in
apps:
- ensuring that widgets without names have labels attached (I believe
Dobey's LDTP-based testsuite for a11y does this already)
- looking at XYWH of widgets and ensuring that tabbing order is sane.
- check that correct ATK events fire when events happen e.g. do radio
buttons in a group change state appropriately when one is selected

Some points were raised for improvement of dogtail:
- move website from current insane URL
( http://people.redhat.com/zcerza/dogtail/ to somewhere less insane,
preferably not in the .redhat.com domain (also, people didn't realize
that the code was on GNOME CVS, and bugs in GNOME bz); make project less
directly associated with Red Hat?  dogtail.org?  live.gnome.org?
- documentation is lacking.  we should add some links to the articles
Len wrote to our website:
http://www.redhat.com/magazine/020jun06/features/dogtail/
http://www.redhat.com/magazine/021jul06/features/dogtail/
http://www.redhat.com/magazine/022aug06/features/dogtail/
or even embed this HTML content in the module's documentation
- can the recorder inject "sleep" statements to reflect actual timings?
- should we port to pyorbit? (aka how Orca works, see diagram above)  In
theory this would give us richer exception support, and lose the
probable bugs in cspi and pyspi - the fewer modules the better (makes
ASCII-art easier too).  IIRC Zack tried this and exchanged one set of
bugs for another - Zack, is this on a branch somewhere?
- how will we integrate with jhbuild?  should dogtail have its own
harness?  or should it be designed to work in an arbitrary harness?
Apparently there's a patch sitting in BZ that implements LDTP and
Dogtail support for jhbuild.

We started hacking, and identified a couple of issues for Dogtail on
Solaris: patch authored for Solaris support in Dogtail and submitted to
GNOME BZ; fixed the other issue (Solaris packaging), and then ran out of
time.

Thanks to all the participants - did I miss anything?
Dave