Test reports and commit policies
- From: Tim Janik <timj imendio com>
- To: Gtk+ Developers <gtk-devel-list gnome org>
- Subject: Test reports and commit policies
- Date: Thu, 20 Dec 2007 17:51:43 +0100 (CET)
A quick update on the unit test reports, a script for test report
generation has now been comitted to GLib and will be used to generate
HTML reports for the test report rules test-report, perf-report and
The reports should render in all browsers and support colorization and
Example reports generated from today's SVN are provided here:
Percentages are reported for the amount of tests that failed or succeeded,
e.g. gtk+/tests/.libs/lt-objecttests is reported with a result of 80.39%,
because ca. 20% of our automated widget property tests fail (generate some
warnings or crash Gtk+, volounteers to look into those cases highly needed).
It'd be nice if someone in the community could set up a build machine for
jhbuild/modulesets/gtk.modules that builds, runs test-report, perf-report
and full-report and uploads the resulting test-report-html, perf-report.html
and full-report.html from GLib and Gtk+ to some public location.
About commit policies, now that we have a test framework in place that
distinguishes between slow (make full-report) and fast (make check) tests,
and allows subcomponent related tests (e.g. make check -C gtk+/gtk/), it'd
be nice to actually make ue of it.
So i'd like to suggest that we establish some basic commit/test
1) Before committing changes in glib/<subdir> or gtk+/<subdir>, a
developer needs to ensure that make check -C glib/<subdir> (or
make check -C gtk+/<subdir>) passes.
2) For major changes (adding new components, changing lots of files,
changing Makefiles, etc.), developers should run make full-report
on the whole module and investigate the failing test cases before
The exact rules/process is open for discussion of course, the above
is meant as some initial food for thought. Please discuss! ;-)
] [Thread Prev