Re: GNOME's testing strategy for GUIs

[Brad Taylor <brad getcoded net>]

Hi Willie,

First off, in the spirit of full disclosure, I should mention that I
work for Medsphere, the company who backs the development of the GPL
testing tool, Strongwind[1].

> I might be opening a big can of worms with this question, and I 
> apologize if someone is already working in this space and I just don't 
> know it.

Ken VanDine and I met at SCALE this last week and discussed some plans
regarding automated testing in GNOME.  We identified a few rough spots:


* Many tools are available for automated testing, and GNOME needs to
  annoint one with holy gnome pee so that it will become an acceptable
  dependency for development.  Whether this is LDTP, Dogtail, Strongwind,
  or others, a clear decision needs to be made so that people can go
  forth with creating tests.

  Having a mix of testing tools in use will only make it more difficult
  for folks to contribute to, and maintain a large body of tests.
        
* Someone needs to step up and write good, thorough documentation for
  developing tests.  To my knowledge, no current testing tool has this.

* A culture of automated testing needs to be developed and fostered
  inside of GNOME so that maintainers feel real benefit in maintaining
  a high-quality test library.  Maintenance is not a trivial thing.

* Distributions who already have automated tests for GNOME applications
  need to have a place to contribute and maintain these tests
  _publicly_, and have them shared by all.  This was made clear in
  discussions with folks at this year's GNOME Summit.


Obviously, there is a lot of work that needs to be done, but these
challenges are by no means insurmountable.

The first thing that needs to come into the picture are good docs.  I've
commited myself to contribute a Gnome Journal article on writing
Strongwind tests as a first start into this field.
 
Cheers,

-Brad

--
[1] http://medsphere.org/projects/strongwind/