Re: jhbuild continuous integration testing: status and plans

On Fri, Mar 8, 2013 at 12:48 AM, Martin Pitt <martin pitt ubuntu com> wrote:
Hello fellow GNOMErs,

after the first round of discussions a month ago[1] about the jhbuild
CI building/testing [2] I'd like to give some status update.

Along the same topic, I wanted to bring to light something that
was recently raised in EDS bugzilla [0].

Vadim filed this bug, and apparently copied the makefile
from another module... I was wondering if standardizing
this would be interesting for the Continuous Integration
servers... the html output it creates is certainly interesting
to have, an example of the output can be found here [1].

My thoughts were mostly that in that proposed patch, the
included Makefile is a bit crack and hacked out, I would
be interested in standardizing that a bit.

My thoughts were:

  a.) Create an m4 macro implementing the gtester rules
       which could be included in gnome-common and easy
       to integrate into various GNOME modules

  b.) Perhaps extend the .doap format (who defines the
       .doap format anyway ?).. so that the build server
       (or jhbuild directly perhaps) could introspect whether
       a module supports the 'make gtester-report' target.

       In that way, the CI server could automatically collect
       these reports only for modules which support them.

Is it interesting for more modules to implement the more fine grained report ?

Did I miss some important information (i.e. is this already done somewhere
at least partially and I'm not aware of it) ?

Any additional thoughts on this ?



Since then, Jean-Baptiste has worked quite a bit on the scripts:
Updating git checkouts as well as building modules are now
parallelized (building still respects the dependency tree), so that we
can now rebuild all the 160 modules in something like 15 minutes now.
This brings us much closer to useful commit-level testing. It now also
uses "jhbuild sysdeps" and a much smaller hardcoded list of extra
build dependencies (which are not exposed by the module lists).
If a build or test fails it now uses jhbuild's -C option to restart
with a clean checkout, to avoid tripping over build system cruft from
autotools file changes.  I spent some time chasing down long-standing
failures in some modules, as well as filing bugs for newly identified

Since the announcement, the system has stabilized a lot, and the set
of test failures is now quite stable.

The thing that hurts most currently is that the machine is behind a
proxy.  This causes quite a lot of test failures (like libgdata's
youtube test), as well as spurious build failures like [3]. We do plan
to move this machine into the DMZ soon, so that it has unrestricted
network access. I'd like to postpone sending out notifications until
that happens, as otherwise we'll spam people too much about irrelevant

Once that happens, we'll set up email notifications for state changes
(e. g. "pass → fails to build", or "test fail → pass") and send them
to Jean-Baptiste and myself first, to give this some more real-world
testing. If that has a low enough noise ratio, we were planning to
mail the maintainers of the affected modules (if the module has a
.doap file), with some hint to notify us if they want to opt out of

Then we can investigate for some time how this works, and debate if
filing bugs would be better or not.

Does that sound like an acceptable plan?



Martin Pitt                        |
Ubuntu Developer (  | Debian Developer  (

desktop-devel-list mailing list
desktop-devel-list gnome org

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]