automated build server setup on EC2

[Colin Walters <walters verbum org>]

Hey,

So over the past few days I've spent a bit of time writing scripts to
automate spinning up the equivalent of http://ostree.gnome.org, inside
EC2.

First, this will help people do builds on their own
hardware/distribution, because you can see how things are set up by
looking at the scripts.

What's way cooler though is that you can implement your own "try
servers" with this; eventually I'd like to get to the point where I can
push a gnomeos-3.6.json manifest which points at wip/foo git branches,
and have the server build it, publish the results, then shut down when
it's done.  There are a variety of details to fill in before this is
possible, but this is a start.

Anyways, the current scripts are targeted for Amazon Linux 2012.03 (a
derivative of Red Hat Enterprise Linux 6).  Basically, here's what you
do to spin up your own server:

* Sign up for AWS
* Log in to the AWS management console
* Click on "Launch Instance"
* Click Amazon Linux, 64 bit
* Click Edit Details
* Change Type to m1.medium (or larger if you prefer, but t1.micro is too
  underpowered)
* Click Storage Device Configuration
* Increase root volume size to 150GB, click Save
* Click Advanced Details
* Paste the contents of
http://git.gnome.org/browse/gnome-ostree/tree/install/amazon-linux-cloud-init.sh
  into the user data field.
* Click launch

Your server should boot up an install everything.

Now, you need to log in via ssh as ec2-user.  On ostree.gnome.org I have
two screen sessions - one for bitbake, an one for ostbuild.

Bitbake screen session:
$ sudo su -
# su - ostree
$ cd ~/build
$ . ~/src/poky/oe-init-build-env gnomeos-build
$ bitbake gnomeos-contents-{runtime,devel}

Ostbuild screen session:
$ sudo su -
# su - ostree
$ ~/.local/bin/jhbuild shell
$ ostbuild resolve --manifest=$HOME/src/gnome-ostree/gnomeos-3.6.json
$ ostbuild build

Both of these above two parts need automation and coordination.  I'm
still on the fence about which CI software to use...leaning towards
BuildBot but I haven't looked at it in detail.

Doing all of this will probably cost you about $7 USD.  That's a full
build from scratch - incremental builds are obviously very fast.

And were we to "productize" this more, we'd probably have a tool to
cache the Bitbake sstate and OSTree repositories in S3 or something, so
that build servers can share data, like how Fedora's Koji has multiple
instances that operate on a shared RPM repository.