[g-a-devel] Role of automated testing frameworks (was Re: Status of IBM a11y)

[Jason White <jasonw ariel its unimelb edu au>]

Here is a quick idea that came to mind after writing my last post.

It has often been said that one of the side-benefits of accessibility support
would be better automated testing of user interfaces. If a user interface can
be read and operated entirely by software, as access for assistive
technologies requires, then it becomes possible to write test scripts that
check the functionality of the application against expected behaviour. That
is, the script interacts with the application via accessibility interfaces and
tests that the correct output is generated.

I know there has been some work carried out in this area, but my question is
whether this could be expanded further. Suppose that a number of "free
software" desktop projects were to develop comprehensive test suites for their
applications using the above method. How effective would this be in detecting
application bugs and avoiding regressions? Would it be worth the effort? One
effect would be that the failure of an "accessibility" interface to yield the
correct result would amount to a test failure of the application, and would
not just be an accessibility problem that may only be discovered and fixed
later.

If this technique turns out to be useful, the benefits would lie in better
software quality generally, and also in improved accessibility support.
Amenability to automated testing could also constitute an additional
motivation for supporting accessibility-related API's early in project
development, and for keeping that support current.

It would also make many accessibility-related bugs visible to application
developers (as test failures) and help to ensure that most of the work
involved in getting accessibility right would lie where it should - with the
authors of the user interface.

What do others think?