Hello fellow GNOME developers,
this already came up as a side issue recently[1], but now we are at a
point where have reasonably stabilized our GNOME jhbuild continuous
builds/integration test server to become actually useful:
https://jenkins.qa.ubuntu.com/view/Raring/view/JHBuild%20Gnome/
This is building gnome-suites-core-3.8.modules, which currently
consists of 160 modules. Builds are updated every 15 minutes, and
triggered whenever there was a new commit in a module or any of its
dependencies. This mostly uses the smarts of jhbuild, we just have
some extra scripts around to pick the results apart for Jenkins and
drive the whole thing [2]. You can click through all the modules, all
their builds, and get their build logs.
Right now there are 151 successes (blue), 5 modules fail to build
(red), and 4 modules build but fail in "make check" (yellow). It's
been like that for a week or two now, so I'd say we are doing
reasonably well for now. Some details:
Build failures:
- colord: recently started depending on libsystemd-login, which we
don't have yet; that's a fault on the Ubuntu side
- e-d-s: calls an undeclared g_cond_timed_wait(), not sure what this
is about
- folks: this started failing very recently, and thus is a perfect
example why this is useful (unqualified ambiguous usage of
HashTable)
- gst-plugins-bad: unknown type GStaticRecMutex; this might be due to
recent changes in streamer? That smells like a case of "broken by
change in dependency, needs updating to new API"
- mutter: worked until Jan 7, now failing on unknown XIBarrierEvent;
that might be a fault in Ubuntu's X.org packages or upstream, I
haven't investigated this yet
Test failures:
- gst-plugins-good, empathy: one test failure, the other tests work
- realmd: This looks like the test suite is making some assumptions
about the environment which aren't true in a headless server?
- webkit: I don't actually see an error in the log; we'll investigate
this closer on our side
This was set up by Jean-Baptiste Lallement, I mostly help out with
reviewing the daily status and cleaning up after some build/test
failures which are due to broken checkouts, stale files, new missing
build dependencies, and so on. It's reasonably maintenance intensive,
but that's something which the two of us are willing to do if this
actually gets used.
The main difference to Colin's ostree builds is that this also runs
"make check", which is one of the main points of this: We want to know
as soon as possible if e. g. a new commit in glib breaks something in
gvfs or evolution-data-server. Where "soon" is measured in minutes
instead of days/weeks, so that the knowledge what got changed and why
is still fresh in the developer's head. That's also why I recently
started to add integration tests to e. g. gvfs or
gnome-settings-daemon, so that over time we can cover more and more
functionality tests in these.
To make this really useful, we can't rely on developers checking this
every hour or every day, of course; instead we need push notifications
as soon as a module starts failing. That's the bit which needs broader
discussion and consent.
I see some obvious options here what to do when the status of a module
(OK/fails tests/fails build) changes:
(1) mail the individual maintainers, as in the DOAP files
(1a) do it for everyone, and let people who don't want this filter
them out on a particular mail header (like "X-GNOME-QA:")
(1b) do this as opt-in
This most often reaches the people who can do something about the
failure. Of course there are cases where it's not the module's fault, but a
dependency changed/got broken. There is no way we can automatically
determine whether it was e. g. a deliberate API break which modules
need to adjust to, or indeed a bug in the depending library, so we
might actually need to mail both the maintainers of the module that
triggered the rebuild, and the maintainers of the module which now
broke.
(2) one big mailing list with all failures, and machine parseable
headers for module/test
This might be more interesting for e. g. the release team (we can
CC: the release team in (1) as well, of course), but will be rather
high-volume, and pretty much forces maintainers to carefully set up
filters.
My gut feeling is that we might start with (2) for a while, see how it
goes, and later switch to (1) when we got some confidence in this?
Opinions most welcome!
Also, I'll gladly work with the developers of the currently failing
modules to get them succeeding. I have full access to the build
machine in case errors aren't reproducible.
Thanks,
Martin
[1] https://mail.gnome.org/archives/desktop-devel-list/2013-January/msg00006.html
[2] http://bazaar.launchpad.net/~jibel/charms/raring/jhbuild/trunk/files
--
Martin Pitt | http://www.piware.de
Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org)
_______________________________________________
desktop-devel-list mailing list
desktop-devel-list gnome org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list