Re: Build sheriffs for GNOME

[Emmanuele Bassi <ebassi gmail com>]

Hi;

On 21 January 2016 at 18:19, Michael Catanzaro <mcatanzaro gnome org> wrote:

> The next step is to convert either jhbuild or gnome-continuous to use
> the others' modulesets. Currently the jhbuild modulesets are the
> official "what we release,"

"What we release" and "what we actually run" are two fairly difference concepts.

I'd contend that the GNOME Continuous manifest is actually much more
close to what distributions ship, and thus what we run on a daily
basis, than what we put into the configure arguments when we
distcheck, or the arguments we put into the moduleset. Mostly, because
nobody, except the release team, smoketest a system composed by
jhbuild, whereas Continuous does smoketesting on a fully composed file
system image.

Continuous also builds the whole stack down to systemd, whereas
JHBuild does not.

JHBuild is meant to be assisted and attended by a human being;
Continuous is mostly meant for automation.

There's also the issue that the JHBuild modulesets cater to a fairly
different crowd, and have flexibility to build on OSX or Windows, with
conditionals and suggested packages.

We could make JHBuild read the Continuous manifest, and start from a
specific module, but the two systems are *not* interchangeable. Each
one has its own requirements and goals, and confusing the two is going
to leave use cases out.

We should certainly ensure that what goes into JHBuild also goes into
Continuous, and possibly with minimal fuss. Ideally, we should move
the JHBuild modulesets and the Continuous manifest out of their
respective repositories, and into a shared location.

> and they are more important because they
> affect the ability of new contributors to build GNOME, so that's what I
> focus on maintaining. Continuous is useful to catch many build
> failures, but it misses many, many more failures than could only be
> detected were it to use the official modulesets.

If Continuous fails to catch build failures then you should update the
manifest until you trigger them. That's the whole point of having
automated builds.

> Continuous is a great first step (the response time is *stunningly*
> good) and we're better off now that we have it, but it's not good
> enough as it's not testing the build tool we use for development, it's
> not testing what we release.

If you're releasing something different than what you have in master
then you're doing something impressively wrong.

You (and by "you" I mean "maintainers") should check what flags and
dependencies are listed in the JHBuild moduleset and which ones are in
the Continuous manifest, and try to make them close or identical.

Bugs will happen, and I consider keeping the Continuous manifest up to
date also part of the job description of a build sheriff.

> >  Many are not even on
> > #gnome-hackers, which means they miss the notification that the
> > something broke the build.
>
> Anecdote: I am on #gnome-hackers so you can ping me, but I don't turn
> on notifications for every message in the room (too many!),

The IRC bot does not notify you personally. You can add a notification
on a specific substring, like 'continuous:build * failed'; or you
could just look at the red message that says that the build failed.
:-)

> so it's
> rare that I notice a failure in one of my modules unless I happen to be
> looking at #gnome-hackers at the time, or someone else pings me. It
> would be really awesome if Continuous could learn to guess who to blame
> for breakage and include the IRC name in the channel;

As I said, it can be incredibly hard to determine that from an
automation standpoint.

Ciao,
 Emmanuele.

-- 
https://www.bassi.io
[ ] ebassi [ gmail com]