Re: Gtk+ unit tests (brainstorming)

[Tim Janik <timj imendio com>]

On Wed, 25 Oct 2006, Federico Mena Quintero wrote:

> On Wed, 2006-10-25 at 17:52 +0200, Tim Janik wrote:

> > - GLib based test programs should never produce a "CRITICAL **:" or
> >    "WARNING **:" message and succeed.
>
> Definitely.  I have some simple code in autotestfilechooser.c to catch
> this and fail the tests if we get warnings/criticals.

oh? hm, looking at the code, it doesn't seem to be significantly superior 
to --g-fatal-warnings:
      fatal_mask = g_log_set_always_fatal (G_LOG_FATAL_MASK);
      fatal_mask |= G_LOG_LEVEL_WARNING | G_LOG_LEVEL_CRITICAL;
      g_log_set_always_fatal (fatal_mask);
or am i missing something?
note that the error counting in your code doesn't funciton beyond 1, because
g_log always treats errors as FATAL, that behaviour is not configurable.

> > 5- simple helper macros to indicate test start/progress/assertions/end.
> >     (we've at least found these useful to have in Beast.)
>
> For the file chooser tests, I just return a boolean from the test
> functions, which obviously indicates if the test passed or not.  What
> sort of macros are you envisioning?

just a very few and simple ones that are used to beautify progress 
indication, e.g.:

TSTART(printfargs)	- print info about the test being run
TOK()			- print progress hyphen unconditionally: '-'
TASSERT(cond)		- print progress hyphen if cond==TRUE, abort otherwise
TCHECK(cond)		- like TASSERT() but skip hyphen printing
TDONE()			- finish test info / progress bar

for simple tests, i just use TSTART/TASSERT/TDONE, but for tests that explore
a wide range of numbers in a loop, a combination of TCHECK/TOK works better
to avoid excess progress indication. e.g.:

  TSTART ("unichar isalnum");
  TASSERT (g_unichar_isalnum ('A') == TRUE);	// prints hypen: '-'
  TASSERT (g_unichar_isalnum ('?') == FALSE);	// prints hypen: '-'
  for (uint i = 0; i < 1000000; i++)
    {
      gunichar uc = rand() % (0x100 << (i % 24));
      if (i % 20000 == 0)
        TOK();					// prints hypen: '-'
      gboolean bb = Unichar::isalnum (uc);
      gboolean gb = g_unichar_isalnum (uc);
      TCHECK (bb == gb);			// silent assertion
    }
  TDONE();

produces:

unichar isalnum: [----------------------------------------------------]

if any of the checks fail, all of __FILE__, __LINE__, __PRETTY_FUNCTION__
and cond are printed of course, e.g.:
** ERROR **: strings.cc:270:int main(int, char**)(): assertion failed: g_unichar_isalnum ('?') == true

having java-style TASSERT_EQUAL(val1,val2) could probably also be nice,
in order to print the mismatching values with the error message.


> > - try setting & getting all widget properties on all widgets over the full
> >    value ranges (sparsely covered by means of random numbers for instance)
> > - try setting & getting all container child properties analogously
>
> I believe the OSDL tests do this already.  We should steal that code;
> there's a *lot* of properties to be tested!

not sure what code you're referring to. but what i meant here is code that
generically queries all properties and explores their value range. this
kind of generic querying code is available in a lot of places already (beast,
gle, gtk prop editor, libglade, gtk-doc, LBs, etc.)


> > - for all widget types, create and destroy them in a loop to:
> >    a) measure basic object setup performance
> >    b) catch obvious leaks
> >    (these would be slowcheck/perf tests)
>
> Yeah.  GtkWidgetProfiler (in gtk+/perf) will help with this.  Manu
> Cornet has improved it a lot for his theme engine torturer, so we should
> use *that* version :)

hm, i really have not much of an idea about what it does beyond exposing
the main window in a loop ;) 
gtk+/perf/README looks interesting though, are there any docs on this
beyond that file?

>  Federico

---
ciaoTJ