jhbuild continuous integration testing: status and plans



Hello fellow GNOMErs,

after the first round of discussions a month ago[1] about the jhbuild
CI building/testing [2] I'd like to give some status update.

Since then, Jean-Baptiste has worked quite a bit on the scripts:
Updating git checkouts as well as building modules are now
parallelized (building still respects the dependency tree), so that we
can now rebuild all the 160 modules in something like 15 minutes now.
This brings us much closer to useful commit-level testing. It now also
uses "jhbuild sysdeps" and a much smaller hardcoded list of extra
build dependencies (which are not exposed by the module lists). 
If a build or test fails it now uses jhbuild's -C option to restart
with a clean checkout, to avoid tripping over build system cruft from
autotools file changes.  I spent some time chasing down long-standing
failures in some modules, as well as filing bugs for newly identified
regressions.

Since the announcement, the system has stabilized a lot, and the set
of test failures is now quite stable.

The thing that hurts most currently is that the machine is behind a
proxy.  This causes quite a lot of test failures (like libgdata's
youtube test), as well as spurious build failures like [3]. We do plan
to move this machine into the DMZ soon, so that it has unrestricted
network access. I'd like to postpone sending out notifications until
that happens, as otherwise we'll spam people too much about irrelevant
issues.

Once that happens, we'll set up email notifications for state changes
(e. g. "pass → fails to build", or "test fail → pass") and send them
to Jean-Baptiste and myself first, to give this some more real-world
testing. If that has a low enough noise ratio, we were planning to
mail the maintainers of the affected modules (if the module has a
.doap file), with some hint to notify us if they want to opt out of
notifications.

Then we can investigate for some time how this works, and debate if
filing bugs would be better or not.

Does that sound like an acceptable plan?

Thanks,

Martin

[1] https://mail.gnome.org/archives/desktop-devel-list/2013-February/msg00025.html
[2] https://jenkins.qa.ubuntu.com/view/Raring/view/JHBuild%20Gnome/
[3] 
https://jenkins.qa.ubuntu.com/view/Raring/view/JHBuild%20Gnome/job/jhbuild-amd64-libgee/92/artifact/libgee.log
-- 
Martin Pitt                        | http://www.piware.de
Ubuntu Developer (www.ubuntu.com)  | Debian Developer  (www.debian.org)

Attachment: signature.asc
Description: Digital signature



[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]