Re: Unit test framework for gnome-games?

[Callum McKenzie <callum spooky-possum org>]

On Sun, 2006-10-29 at 15:43 +0100, Andreas Røsdal wrote:
> I've been thinking about using a unit test framework for gnome-games,
> to automate testing and allow bugs to be detected early. In particular I 
> think it would be useful to have a testing framework that would emulate 
> playing through all the games, to test typical usage and detect crashes. 
> This would hopefully lead to more stable games, reports of test-coverage 
> and allow regression testing[1]. For example, we could potentially 
> enforce that all new patches should also implement a unit test.
> 
> A suggestion would be to use Check[2] for the games written in C,
> and PyUnit[3] for the two games using Python. Are there other 
> suggestions, and is using a unit testing for gnome-games a sane idea?  :)
> 

PyUnit is just fine for python stuff (not including GUI stuff, I have no
idea how to do that). 

Is it sane? When I have used unit tests I've found them very useful. The
question is whether there are parts of the games which are suitable for
automated testing: the games tend to include a lot of global variables
and variables with a lot of state. "Library" routines are easy to write
tests for.

Another question is what to test. At simplest, you could just add tests
as bugs are found to stop them reappearing later. Writing comprehensive
tests for the "library" routines is also useful: it can find obscure
bugs if you test the functions with extreme and odd values.
Unfortunately this sort of test takes a lot of time to write if you want
any decent sort of coverage. Fortunately coverage doesn't have to be
anywhere near complete for it to be useful.

 - Callum