Re: GNOME's testing strategy for GUIs

["Luis Villa" <luis tieguy org>]

On Feb 16, 2008 5:00 PM, Luis Villa <luis tieguy org> wrote:
> On Feb 16, 2008 4:50 PM, Willie Walker <William Walker sun com> wrote:
> > > I'd like to see documentation on 'GNOME recommended automated testing'
> > > for all the kinds of projects we see in GNOME (including for the various
> > > languages). I think this thread is a great way to try and get community
> > > consensus and to collect information on what various projects use. I
> > > suspect a lot of projects use none or very little (sadly, including GOK).
> > >
> > > IMHO this needs to change.
> >
> > Agreed.  We tried to get some momentum back at GNOME Boston 2006
> > (http://live.gnome.org/TestingUsingAtSpi), but we never really gained
> > traction.  It may that the community wasn't ready then, but it might be
> > ready now.
>
> As I suggested to Willie earlier in private mail, I think the key is
> getting something actually used. Get someone to run whatever tests
> there are on a regular, automated basis, and file bugs as a result. It
> will be imperfect (very imperfect to start with, owing to issues with
> the tools, lack of test coverage, etc.) but it will be better than
> nothing. That will:
>
> * create general developer awareness
> * encourage people to write tests, because they will know that the
> tests will be executed/used
> * encourage developers of the test harnesses to improve based on
> real-world feedback, which is obviously pretty lacking right now
> (though there appear to be some hints that it is happening)
>
> The rest (documentation, etc.) falls out of actually using it, IMHO.

I might add that, when I last looked at the problem (a looooooong time
ago, so take with a very large grain of salt) I felt that the right
thing to do was to integrate this with 'make test' and run it as part
of the jhbuild/tinderbox process, so as to get it the broadest
possible use. But there may be other approaches that make the most
sense.

Luis