Re: automated build server setup on EC2

[Colin Walters <walters verbum org>]

On Mon, 2012-08-20 at 12:10 +0300, Adrian Perez de Castro wrote:

> Ouch, seeing this hurts a bit, hehe ;-)
> 
> A shorthand that makes “sudo” *both* switch user and launch a login
> shell in a single command would be (with the added bonus of creating
> less intermediate processes):
> 
>   # sudo -u ostree -i

Ah, thanks, I didn't know about that.

> Personally I like BuildBot [1] and have already some experience setting
> up both the master server and the workers. Being Python makes it quite
> easily hackable.

The problem is BuildBot is just not designed to build multiple
repositories as part of one whole.  As I understand it, the
jhbuild-buildbot integration that Igalia (right?) did is somewhat messy
due to this.  You can have *independent* builds feeding into one
waterfall - that's what BuildBot is designed for.  But that's not how
GNOME works.

And the thing is - even if we did have independent builds, I actually
disagree that's the right approach.  Basically we shouldn't care whether
evolution-data-server builds or doesn't build - we should care whether
GNOME *as a whole* works.  So it's actually a *good* thing if a break in
any component fails the whole build.

One might ask: Wouldn't this result in the build falling over
constantly?  The last 4 months of real-world experience with the current
system is that that's not the case.  Really there aren't that many
commits going into GNOME.


So, this weekend I was thinking that we could switch to git submodules
instead of the gross/hacky snapshot.json files.  That'd be a win on two
levels; first we'd only have *one* git repository to perform CI on, and
second we'd be able to reuse the various tools that exist for git
submodules.

For example, I had started writing an "ostbuild git grep" type tool
to search across the source code, but that'd be entirely unnecessary
if gnome-ostree had everything else as submodules.

> IMHO we could set it up watching the “gnome-ostree” Git repository,
> doing builds automatically whenever a commit reaches the “master”
> branch. 

In the *current* system though we need to have something with does
"ostbuild resolve --fetch" periodically.  Or more ideally, it's driven
by push notifications from git.gnome.org and git.freedesktop.org etc.

There's also the question of how we handle the Poky/Yocto buildroot.

The main questions with submodules are:

1) Does it live in gnome-ostree?  Probably.
2) Is it OK if for every commit to modules gnome-ostree tracks, we
   have an automated commit to update the submodule?  Would add to
   the noise on #commits, but maybe we can filter it out.

Thoughts?