notes from QA BOF today

[Luis Villa <luis villa gmail com>]

So, we had a QA BOF at the summit. Because the dogtail guys were
there, and they are very interested in getting dogtail into a jhbuild
tinderbox, that became most of the thrust of the BOF. A dump of my
notes:

things we're blocking on to run dogtail tests in tinderbox:
* makefile-without-configure module for pyspi/dogtail in jhbuild or
track down an autotools guru who can be volunteered (or 'touch
autogen.sh; cvs commit autogen.sh' ;)
* create dogtail-tests/ to contain module-specific tests and
meta-tests. should tag this with same patterns as gnome for gnome
releases, so you can check out the 2.14 tests forever
* create a test module type in jhbuild (or just do same crappy
autogen/configure hack)
* running headless: figure out xvfb v. xfake, maybe just use whichever
is installed; need to get a full session going- what exactly are the
needs for the session? could jhbuild run gnome-session (inside session
run test scripts) or could maybe do the nat-jeff session:
http://nat.org/2005/october/#Keep-It-Simple-Stupid instead of a real
session

things we'd really like:
 * recording tools that record scripts for maintainers to play back
(for reproduction purposes and regression testing) and potentially
also video. Could do a sabayon-style editor to trim extraneous things.
Probably need to require playback/verification/duplication before
upload.
* 'real' jhbuild integration- do it per module with separate column
instead of fake modules; this might require quite a bit of jhbuild
rearch, depending on exact goals.

performance/memory/etc. stuff:
* get the infrastructure running need to talk to the performance
people; tell them 'we're going to be running all the apps in an
automated fashion, what do you want to know?' examples include:
bootchart.org, valgrind