Re: GNOME's testing strategy for GUIs

["Luis Villa" <luis tieguy org>]

On Feb 16, 2008 4:50 PM, Willie Walker <William Walker sun com> wrote:
> > I'd like to see documentation on 'GNOME recommended automated testing'
> > for all the kinds of projects we see in GNOME (including for the various
> > languages). I think this thread is a great way to try and get community
> > consensus and to collect information on what various projects use. I
> > suspect a lot of projects use none or very little (sadly, including GOK).
> >
> > IMHO this needs to change.
>
> Agreed.  We tried to get some momentum back at GNOME Boston 2006
> (http://live.gnome.org/TestingUsingAtSpi), but we never really gained
> traction.  It may that the community wasn't ready then, but it might be
> ready now.

As I suggested to Willie earlier in private mail, I think the key is
getting something actually used. Get someone to run whatever tests
there are on a regular, automated basis, and file bugs as a result. It
will be imperfect (very imperfect to start with, owing to issues with
the tools, lack of test coverage, etc.) but it will be better than
nothing. That will:

* create general developer awareness
* encourage people to write tests, because they will know that the
tests will be executed/used
* encourage developers of the test harnesses to improve based on
real-world feedback, which is obviously pretty lacking right now
(though there appear to be some hints that it is happening)

The rest (documentation, etc.) falls out of actually using it, IMHO.

And by the way, I'm glad this discussion is happening- it is a
*hugely* critical issue, I believe, and if the various distros/OSes
collaborated to make this happen, would pay off at at least that 1:100
ratio mentioned earlier in terms of improved quality and robustness.

Luis