Re: New GnomeGoal proposal: InstalledTests

[Colin Walters <walters verbum org>]

On Fri, 2013-04-26 at 08:46 -0400, Matthias Clasen wrote:

> You are not going to get me to buy eagerly into a new installed tests
> scheme for glib if it means that I have to give up make check.

Well, would you be OK with:

$ jhbuild make
$ gnome-desktop-testing-runner glib

?  

Note though that not all the GLib tests are of the installed form;
for example, glib/tests/1bit-mutex.c actually includes some of the GLib
source code again and rebuilds it in a different configuration.

That sort of thing is what I'd call a "white box unit test".  It knows
about (and reuses) the implementation details.  But most of GLib's test
are black box, and those are the sort that make sense to install,
because that's the best way to mimic a black box scenario.

But anyways I don't plan to do anything with GLib soon until I've made
Giovanni happy again in:

https://bugzilla.gnome.org/show_bug.cgi?id=698935

You have to understand the win here though - if we install the tests,
they can be easily fully automated.  Since I've switched the gjs test
suite to be installed, it's already run over 28 times just so far today
(UTC), without any human intervention.

Also, debugging installed tests is *so* much nicer than "make check" -
you don't have to fight through layers of generated Makefile and libtool
wrapping your actual binary if you want to run gdb/valgrind on it.