Re: Gtk+ unit tests (brainstorming)

[Iago Toral Quiroga <itoral igalia com>]

El mié, 15-11-2006 a las 10:51 +0100, Iago Toral Quiroga escribió:
> > > I'll add here some points supporting Check ;):
> > 
> > ok, adressing them one by one, since i see multiple reasons for not
> > using Check ;)
> > 
> [...]
> > it's not clear that Check (besides than being an additional
> dependency in
> > itself) fullfils all the portability requirements of glib/gtk+ for
> these
> > cases though.
> > 
> [...]
> > - Check may be widely used, but is presented as "[...] at the moment
> only
> >    sporadically maintained" (http://check.sourceforge.net/).
> >    that alone causes me to veto any Check dependency for glib/gtk
> already ;)
> 
> I've just asked Chris Pickett (Check maintainer) about this issues, so
> he can confirm. I'll forward his opinion in a later mail. 

Here it is:

------

Hi Iago,

Well, I looked at most C unit testing frameworks out there, and ended up
settling on Check.

That I wrote Check is sporadically maintained on the web page means that
it has reached a point where it is fairly stable, and it does the job 
well for people that use it.  You can quote me on that if you want (in 
fact, I'm going to update the web page).  There are lots of users, a 
very low-volume mailing list, and not very many open bugs.  Search 
Google for srunner_create or something and you will see what I mean.

It has some problems with failing its own unit tests at the moment when 
built, but I think it has to do with some hard-coded timeout in the unit
tests and the speed of newer processors.  I haven't actually encountered
any problems using it myself.

As for portability, I don't think there are any serious issues, I 
remember seeing a Windows patch somewhere.  I recently updated it to 
Autoconf 2.50+ and switched the documentation from DocBook to Texinfo.

I know people love to rewrite the world, but in this case, I would 
recommend just using Check for now, and if any problems are encountered,
then write some throwaway scripts to convert the tests to a new format, 
or fix what's actually broken.  It can't be difficult to do either way, 
and I think it would save a lot of time.  Right now, they're proposing a
big speculative design before knowing through experience what their 
needs for GTK+ really are, and experience will help a lot: either they 
will say, "Oh, hey, Check is actually great!" or they will say, "Damn, 
these are all the things that sucked about Check and let's make sure we 
get them right this time!"

Probably my best general advice is not to write too many tests, and not 
to go crazy testing assertions, but that has nothing to do with Check. 
You might want to get gcov working at the same time.

You might also do well to send some emails to other projects, e.g. 
GStreamer, and find out what they think of it, whether they would 
recommend it, etc. etc.  I'd be interested to know what they say.  Hmm, 
I just looked at email #3 and I see Stefan from GStreamer recommending 
Check.  Well, I would just listen to him, quite frankly.

Cheers,
Chris

Iago Toral Quiroga wrote:
> Hi Chris,
> 
> there is some debate in GTK+ about unit testing. I tried to convince
> people to use Check as framework but it seems they prefer doing
> something from scratch.
> 
> The main points against Check that they have provided are:
> 
>    * The web page (check.sourceforge.net) states that Check is
> sporadically maintained.
> 
>    * It's not clear that Check fullfils all the portability
requirements
> of glib/gtk+.
> 
>    * They think the functionality needed to fulfill glib/gtk+ testing
> needs can be implemented ad-hoc with not too much effort, avoiding
> another dependency.
> 
> What's your opinion about this issues?
> 
> Just in case you want to follow the debate:
>
http://mail.gnome.org/archives/gtk-devel-list/2006-October/msg00093.html
>
http://mail.gnome.org/archives/gtk-devel-list/2006-October/msg00129.html
>
http://mail.gnome.org/archives/gtk-devel-list/2006-October/msg00167.html
>
http://mail.gnome.org/archives/gtk-devel-list/2006-November/msg00077.html
> 
> (Actually this is a bigger debate than just using Check or not, so I
> wrote only some URLs that touch the Check debate at some point).
> 
> Cheers,
> Iago.