Test reports and commit policies

[Tim Janik <timj imendio com>]

Hey All.

A quick update on the unit test reports, a script for test report
generation has now been comitted to GLib and will be used to generate
HTML reports for the test report rules test-report, perf-report and
full-report.
The reports should render in all browsers and support colorization and
collapsed/popup details for CSS and javascript enabled browsers.
Example reports generated from today's SVN are provided here:
	http://people.imendio.com/timj/examples/glib-full-report.html
	http://people.imendio.com/timj/examples/gtk+-full-report.html

Percentages are reported for the amount of tests that failed or succeeded,
e.g. gtk+/tests/.libs/lt-objecttests is reported with a result of 80.39%,
because ca. 20% of our automated widget property tests fail (generate some
warnings or crash Gtk+, volounteers to look into those cases highly needed).

It'd be nice if someone in the community could set up a build machine for
jhbuild/modulesets/gtk.modules that builds, runs test-report, perf-report
and full-report and uploads the resulting test-report-html, perf-report.html
and full-report.html from GLib and Gtk+ to some public location.

About commit policies, now that we have a test framework in place that
distinguishes between slow (make full-report) and fast (make check) tests,
and allows subcomponent related tests (e.g. make check -C gtk+/gtk/), it'd
be nice to actually make ue of it.

So i'd like to suggest that we establish some basic commit/test
policies, e.g.:

1) Before committing changes in glib/<subdir> or gtk+/<subdir>, a
   developer needs to ensure that make check -C glib/<subdir> (or
   make check -C gtk+/<subdir>) passes.

2) For major changes (adding new components, changing lots of files,
   changing Makefiles, etc.), developers should run make full-report
   on the whole module and investigate the failing test cases before
   committing.

The exact rules/process is open for discussion of course, the above
is meant as some initial food for thought. Please discuss! ;-)

---
ciaoTJ