Re: gtk regression test suite

[muppet <scott asofyet org>]

On Jun 13, 2005, at 4:48 AM, Jon Dowland wrote:

> I've been working on a small patch for GTK+ which changes a
> GtkScrolledWindow property to be a style property (or GtkSetting)[1].
> This has been both to `scratch an itch' and to begin learning my way
> around GTK+ with a view to future hacking.

This is likely to break code that expects that property to be an  
object property.  You never know who is using these things.  Would it  
be reasonable to have the object property be a proxy for the style  
proxy, to retain compatibility?


> I was wondering if there was any interest amongst the GTK+ hackers to
> investigate creating a regression test suite. I think API changes and
> not-API-changes-but-similar (like [1]) could be caught with such a
> framework.

The perl bindings[1] have an extensive API regression test suite[2].   
This suite uses the standard perl test framework with some Gtk2- 
related enhancements (auto-init of gtk+, and some helpers to make  
deferred tests cleaner), and exists mainly to verify that we call the  
bound APIs correctly.  That is, we test the validity of the bindings,  
not the correctness of the bound API itself.

That said, we have, with our test suite, found quite a handful of  
bugs in gtk+ itself (memory leaks, missed assertions, corner-case  
crashes, and even functionality regressions).

So, i can vouch for the usefulness of an automated suite, and point  
to an example of how we set one up.

The basic trick is to know what things require a main loop in order  
to function correctly.  Some things just can't be tested simply.  For  
example, drag'n'drop, or just about anything to do with user input,  
will require help from something that plays back events (at least two  
such projects exist, one in gnome cvs, iirc).  Other things can be  
very easily tricked; for every release of gtk+, we have to repair our  
tests for GtkFileChooser, because the behavior changes slightly under  
the hood (this file used to be selected immediately, but now is  
selected after an idle, etc).


[1] http://gtk2-perl.sourceforge.net/
[2] http://cvs.sourceforge.net/viewcvs.py/gtk2-perl/gtk2-perl-xs/Gtk2/t/

--
Holy crap, dude, we have kids!
    -- Elysse, six days after giving birth to twins