Gnome Flatpak build system, descriptions and questions

[Alexander Larsson <alexl redhat com>]

After some work I now have the gnome runtimes and applications
building again, now on the new gnome build infrastructure.

We no have these nuild machines:

sdkbuilder1.gnome.org - x86_64 & i386, stable & nightly
gnome5.codethink.co.uk - aarch64, nightly
gnome6.codethink.co.uk - aarch64, stable
gnome7.codethink.co.uk - arm, nightly
gnome8.codethink.co.uk - arm, stable

The first one is a VM that us running in the gnome infrastructure, and
the rest are aarch64 blades that collabora are running for the Gnome
project.

All these machines run the build scripts from:
  https://github.com/flatpak/flatpak-build-scripts

They are onfigured to build these repos:
  https://github.com/flatpak/freedesktop-sdk-base/ (branches: 1.4)
  https://github.com/flatpak/freedesktop-sdk-images (branches: 1.4)
  https://git.gnome.org/browse/gnome-sdk-images (branches: gnome-3-20,
master)
  https://git.gnome.org/browse/gnome-apps-nightly (branches: gnome-3-
20, master)

These builds are run continuously every 10 minutes if the
corresponding git repos change (i.e some json manifest changed) and
nightly for a forced build (i.e. the json didn't change in git, but it
pulls the sources from a git repo that got new commits).

The resulting ostree repositories and build logs are then exposed over
http from each build machine. For instance, the x86-64 machine is
available at http://sdkbuilder1.gnome.org/ . Builds results from all
these machines are also announced in #flatpak and #flatpak-builds on
freenode.

These individual repos ar then pulled (using ostree pull) to the
machine that serves the final bits to the users, sdk.gnome.org. The
pull goes to a staging repository, and then a script is run that
re-signs and re-commits each app/runtime, and then creates static
deltas
and updates the summary files for efficient downloads.

The pull+sign+commit for the nightly repo happens in a cron-job, so it
is fully automatic. However, the gpg key for the stable releases
requires manual passwork unlocking, so its done manually by me
as-needed.

So, the basics here is now working again, on the right infrastructure.
Where do we go from here? I have some issues I want to discuss:

sdk.gnome.org is now the same machine as build.gnome.org, which is
doing the gnome-continuous builds. I'm not sure if that is a problem,
but it may be pretty loaded when its building something and at the
same time doing a lot of sdk downloads.

Right now, anyone can commit to the gnome-sdk git repos, but mostly it
is me doing all the work maintaining the gnome runtimes. It would be
nice to have a wider group taking responsibility for this, possibly
including some people from the release team. This would include doing
work on keeping the runtime building, tweaking and testing it, etc, as
well as maintaining the build scripts and manually signing new stable
releases. Who is interested in this?

Currently all the gnome apps are built from a single repository[1]
with a bunch of json files. This makes it very easy to build, as we
just regularly pull that and build everything in it. However, it would
make more sense for each individual application developers to maintain
the manifest in the applications git repo. That makes the build script
more complicated though. I guess we could have a file with names of
the repo+file of everything to build, but we would need someone to
work on the build scripts to make this work. Any takers?

[1] https://git.gnome.org/browse/gnome-apps-nightly

-- 
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
 Alexander Larsson                                            Red Hat, Inc 
       alexl redhat com            alexander larsson gmail com 
He's an oversexed small-town hairdresser who dotes on his loving old ma. 
She's a sarcastic out-of-work wrestler from the wrong side of the tracks. 
They fight crime!