Re: Build sheriffs for GNOME

[SAHIL SAREEN <sahil sareen hotmail com>]

Re: Build sheriffs for GNOME

SAHIL SAREEN
Reply|
To:
Emmanuele Bassi <ebassi gmail com>; Thu 1/21/2016 8:47 PM
Hey ebassi

I feel we should spend time building a simple "build sheriff" (maybe an extension of https://build.gnome.org) 
that auto-files a "cri" bug for the module whose build failed. This should make sure that it isn't creating 
duplicate bugs(reopening the old one if exists) for the same module and append a comment giving the link to 
the respective build failure.

The package maintainers(or the #testable s) should then either revert the bad commits or fix them and close 
the bug.

Thoughts?

-Sahil

________________________________________
From: desktop-devel-list <desktop-devel-list-bounces gnome org> on behalf of Emmanuele Bassi <ebassi gmail 
com>
Sent: Thursday, January 21, 2016 8:24 PM
To: Desktop Development List
Subject: Build sheriffs for GNOME

Hi all;

Many of you know about GNOME Continuous, and build.gnome.org — and for
those who don't, here's two handy links:

  - https://build.gnome.org
  - https://wiki.gnome.org/Projects/GnomeContinuous

In short: we're currently building the core GNOME and some
applications every time something gets committed to git.gnome.org;
we're also doing various tests — like smoketesting the session and
running applications — and building VM images out of the build
process.

This effort led to various benefits, including JHBuild not constantly
being in a broken state, and most projects hosted on git.gnome.org
finally building when builddir != srcdir, which should improve the
ability of maintainers to do `make distcheck` on release days.

What we need now, though, are "build sheriffs", i.e. people that watch
the build process and ensure that it continues smoothly. Ideally,
every maintainer would be on IRC, and would join the #testable
channel; sadly, that's not the case. Many are not even on
#gnome-hackers, which means they miss the notification that the
something broke the build. We cannot always send emails to the last
committer of a broken module because GNOME is a complex project, and a
change in a dependency may indeed break your project, even if you
didn't know about it.

For the past few months a few people, myself included, have been
hunting down build breakages; if we exclude the issues with the build
machine itself throwing a fit — something that usually gets fixed by
Colin kicking it — the vast majority of build breakages come from
issues inside GNOME projects.

What usually happens when a build goes into perma-red (i.e. it keeps
failing over the same component) is that somebody on the #testable IRC
(usually me or Colin Walters) tags the module inside the manifest,
opens a bug, and hopes that a fix get applied and  on the channel so
that the tag gets reverted.

This is not enough, and it does not raise the bar in keeping GNOME in
a buildable state. It actually lowers it a fair , to the effective
point that *nobody* cares about Continuous builds.

I want this to change. I want to be able to revert failing commits on
the offending modules, if they are hosted on GNOME infrastructure, if
they fail for more than N hours, and *then* open a bug about it.
Ideally, I want to tag only modules that are *not* hosted on GNOME
infrastructure, as they are beyond our control and commit
capabilities. In short, I want to ensure that GNOME maintainers become
a bit more proactive in giving a crap about their modules breaking on
something that is not their own computers.

So, here's what will happen in the near future in case of a build break:

  - a build sheriff will identify the component that is actually
breaking; this usually takes about a minute of reading the build log
linked by the IRC bot
  - the build sheriff will try to reach the maintainer of the
component and see if they are on IRC or any other form of IM; I
strongly suggest you either use the same nickname as your GNOME user
id in the DOAP file of your projects, or we'll need to figure out a
way to link your IRC nickname with the project description
  - if nothing happens for a reasonable amount of time (I'd give one
to three hours)
    - if the offending component is not hosted on git.gnome.org, the
build sheriff will tag the component in the gnome-continuous manifest
file
    - if the offending component is hosted on git.gnome.org, the build
sheriff will revert the commits that caused the breakage
  - in either case, a bug will be filed in the bug tracker for the
component, if one is not already open (that's why you should always
list bugs into the commit message)

What does a build sheriff need to know? Usually, breakages involve the
build system. If you know autotools and/or CMake enough to fix issues
with them, you should consider joining. The wiki page on GNOME
Continuous describes the system — so you should take some time to
familiarise yourself with it:

    https://git.gnome.org/browse/gnome-continuous

Especially the manifest file.

The process of keeping GNOME building will be much more smooth and
painless the more people we have watching the Continuous build on
#testable. Ideally, you should *always* look at
https://build.gnome.org or join the #testable IRC channel.

For the time being, Colin Walters, Javier Jardón, and myself will be
watching the build as much as we can spare (we're also busy with our
own jobs ;-)) and try to fix issues; any help is much appreciated,
though.

Ciao,
 Emmanuele.

--
https://www.bassi.io
[ ] ebassi [ gmail com]
_______________________________________________
desktop-devel-list mailing list
desktop-devel-list gnome org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list