Re: automated build server setup on EC2

Colin Walters <walters verbum org> writes:

>> Personally I like BuildBot [1] and have already some experience setting
>> up both the master server and the workers. Being Python makes it quite
>> easily hackable.
> The problem is BuildBot is just not designed to build multiple
> repositories as part of one whole.  As I understand it, the
> jhbuild-buildbot integration that Igalia (right?) did is somewhat messy
> due to this.  You can have *independent* builds feeding into one
> waterfall - that's what BuildBot is designed for.  But that's not how
> GNOME works.

Dunno how the jhbuild+buildbot integration was done, I was not around
at the time. I gave a quick glance to the jhbuild code, and the
“jhbuild bot” commands use Buildbot's Python modules but as you mention
some of the parts seem to be a bit flaky (for example the parsing of
the GNOME commits-list to check which modules to rebuild comes to mind).

> And the thing is - even if we did have independent builds, I actually
> disagree that's the right approach.  Basically we shouldn't care whether
> evolution-data-server builds or doesn't build - we should care whether
> GNOME *as a whole* works.  So it's actually a *good* thing if a break in
> any component fails the whole build.

Agreed. I'm working in making full builds with yocto+ostbuild+buildbot,
as a whole. More details in a upcoming mail to gnomeos-list :)

> One might ask: Wouldn't this result in the build falling over
> constantly?  The last 4 months of real-world experience with the current
> system is that that's not the case.  Really there aren't that many
> commits going into GNOME.

Taking GNOME “as a whole” and trying to rebuild it “as a whole” instead
of whenever there are commits of individual modules is indeed much less

> So, this weekend I was thinking that we could switch to git submodules
> instead of the gross/hacky snapshot.json files.  That'd be a win on two
> levels; first we'd only have *one* git repository to perform CI on, and
> second we'd be able to reuse the various tools that exist for git
> submodules.

Not sure what you mean with using Git submodules... Would that be using
a Git submodule per component built, each one containing a .json and
the needed patches with to make the module (component?) build?
(Probably I'm not following, sorry :-/)

>> IMHO we could set it up watching the “gnome-ostree” Git repository,
>> doing builds automatically whenever a commit reaches the “master”
>> branch. 
> In the *current* system though we need to have something with does
> "ostbuild resolve --fetch" periodically.  Or more ideally, it's driven
> by push notifications from and etc.

Initially I plan to *always* include an “ostbuild resolve --fetch” step,
to get the basiscs working, but indeed it would be much better to have
push notifications (the post-receive hook) at some point.

> [...]
> 2) Is it OK if for every commit to modules gnome-ostree tracks, we
>    have an automated commit to update the submodule?  Would add to
>    the noise on #commits, but maybe we can filter it out.

Probably yes, filtering them out.


Adrian Perez <aperez igalia com> - Sent from my toaster
Igalia - Free Software Engineering

Attachment: pgpMBuCoaYqVA.pgp
Description: PGP signature

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]