Re: notes from QA BOF today

[Nagappan <anagappan novell com>]

Hi Dave,

Dave Malcolm wrote:
> On Mon, 2005-10-10 at 13:10 -0400, Luis Villa wrote:
>   
> > So, we had a QA BOF at the summit. Because the dogtail guys were
> > there, and they are very interested in getting dogtail into a jhbuild
> > tinderbox, that became most of the thrust of the BOF. A dump of my
> > notes:
> >
> > things we're blocking on to run dogtail tests in tinderbox:
> > * makefile-without-configure module for pyspi/dogtail in jhbuild or
> > track down an autotools guru who can be volunteered (or 'touch
> > autogen.sh; cvs commit autogen.sh' ;)
> >     
>
> I've created bugs 318534 and 318535 to cover the unjhbuildability  (if
> that's a word) of pyspi and dogtail respectively.
>
> I've created bug 318531 as a tracker bug for getting tinderbox building
> working.  This has the above two bugs as blockers (I like tracker bugs).
> I've attached a patch to the gnome-2.14.modules file to 318531 which
> adds pyspi and dogtail (is this committable, or will this break the
> build for everyone?).  
>
>   
> > * create dogtail-tests/ to contain module-specific tests and
> > meta-tests. should tag this with same patterns as gnome for gnome
> > releases, so you can check out the 2.14 tests forever
> >     
>
> I've imported this into GNOME CVS, with an initial testcase (which has
> gcalctool doing multiplication tables).  Doesn't yet work, due to the
> nonjhbuildability of dogtail.
>
> I also attached a proposed "gnome-2.14-testsuite.modules" file to 318531
> which contains this testcase as a module.  I've deliberately kept the
> name agnostic regarding which test automation system is in use - it can
> contain both LDTP and Dogtail scripts, depending on what people feel
> comfortable working with.
>   
Thats interesting :)
>   
> > * create a test module type in jhbuild (or just do same crappy
> > autogen/configure hack)
> >     
>
> jhbuild seems to happily deal with the lack of any autotool files; my
> testcase module has a Makefile with a simple:
>
> all:
>  	name-of-test.py
>
> layout, which succesfully invokes the test in jhbuild when I run
> jhbuild buildone dogtail-tests/gcalctool
>
> It invokes the test script, but fails due to it not finding dogtail
> installed in the correct python location.
>
>   
> > * running headless: figure out xvfb v. xfake, maybe just use whichever
> > is installed; need to get a full session going- what exactly are the
> > needs for the session? could jhbuild run gnome-session (inside session
> > run test scripts) or could maybe do the nat-jeff session:
> > http://nat.org/2005/october/#Keep-It-Simple-Stupid instead of a real
> > session
> >
> >     
>
> I filed this as bug 318537 and added it to the tracker bug.
>   
I have done some work on that based on Luis Villa suggestion - 
http://gnomebangalore.org/ldtp/index.php/Howto_run_pyldtp_inside_Xvfb_which_runs_in_xinit_session
>   
> > things we'd really like:
> >  * recording tools that record scripts for maintainers to play back
> > (for reproduction purposes and regression testing) and potentially
> > also video. Could do a sabayon-style editor to trim extraneous things.
> > Probably need to require playback/verification/duplication before
> > upload.
> >     
> I filed this as bug 318538; it's not on the tinderbox tracker bug.
>
>   
> > * 'real' jhbuild integration- do it per module with separate column
> > instead of fake modules; this might require quite a bit of jhbuild
> > rearch, depending on exact goals.
> >     
>
>   
> > performance/memory/etc. stuff:
> > * get the infrastructure running need to talk to the performance
> > people; tell them 'we're going to be running all the apps in an
> > automated fashion, what do you want to know?' examples include:
> > bootchart.org, valgrind
> >     
LDTP already have support for this :) - 
http://gnomebangalore.org/ldtp/index.php/Class_pstats
As of now, this can track only the memory and CPU of the specified 
application, while executing the test-scripts.
> I didn't any bugs for these at this time - the idea of this list is to
> try to reduce the total number of open bugs, right :-P
>
> Dave
>   
Thanks
Nagappan
--
Nagappan A <anagappan novell com>
Novell Software Development (I) Pvt. Ltd.
Linux Desktop Testing Project - http://gnomebangalore.org/ldtp/index.php/Main_Page
http://nagappanal.blogspot.com/