Re: Announcement/RFC: jhbuild continuous integration testing

On Fri, Feb 15, 2013 at 7:57 AM, Olav Vitters <olav vitters nl> wrote:
On Thu, Feb 14, 2013 at 07:12:10AM +0100, Martin Pitt wrote:
That is indeed the long-term plan, but there's still some work to be
done before we can do that. The machine we are running this on has 64
2.7 GHz cores and 64 GB of RAM, that really isn't a bottleneck right
now. The main two problems right now are that the "jhbuild update"
stage takes some 5 minutes to update all the ~ 160 git trees, and
that jhbuild build doesn't parallelize at all, i. e. build modules
which don't depend on each other could build in parallel.

Could you perhaps do a test that does 10 git checkouts at once (real
ones, while things are updated and so on)? I think you might eventually
run into issues with the bandwidth between Red Hat NOC and Canonical

If you see a bottleneck problem, please say so so that it can be worked
on at the same time as parallelizing jhbuild. E.g. there was a plan to
mirror, but there are also a few GNOME servers @
Canonical that could be reused.

Fixing that bottleneck also sounds like one of the first steps towards
setting up a patch queue approach, i.e. set up a local mirror of all
remote git repos which is periodically updated... and fixup jhbuild
to optionally update from those local mirror instead of accessing
remote ones all the time.

Of course, making jhbuild parallelize the actual module builds is
a bit more complex (probably just a bit of python scripting ? not sure).

Another thought, if this gets integrated in a 'mostly a feature of jhbuild'
fashion, this can have some really interesting additional benefits.

Many projects that use the gnome platform libs already do create
their own modulesets, and can then easily set this up on their own
build server for their own project (improving the gnome developer
story in a big/real way), first thing that comes to mind is the osx
builds of Clutter and GTK+ (which are mostly just customized jhbuild
modulesets), not sure about win32... do we use jhbuild inside MSYS
I can't remember ?


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]