GNOME Continous and xdg-app

[Giovanni Campagna <scampa giovanni gmail com>]

Hello all,

At the GNOME Summit this year I had time to make a quick and dirty (but 
working) patch to evolve gnome-continous to produce xdg-apps. The code 
is at https://bugzilla.gnome.org/show_bug.cgi?id=756443, and Colin 
asked me to present it on the ML, so here it is.

What I was trying to do was obtaining a build system for producing 
xdg-apps.
In particular, the major issue with current build systems is that they 
are focused on single packages, but an app is necessarily a composition 
of multiple packages and dependencies bundled in one deliverable. Hence 
the need for a meta-build system, that would mesh the packages togheter.
Various choices are available. One can write spec files and 
debian/control files, but they are cumbersome, long, and generally 
impractical for bundling. One can use jhbuild and write a jhbuild 
adapter to use the xdg-app build enviornment. Or one can use 
gnome-continuous and it's nice declarative JSON format.

I would like to stress that really I'm just reusing the build system 
(ostbuild) here, and only some of it. The goal is not continuous 
integration, rather it is actual final releasing of xdg-app bundles. I 
wanted to prototype something to explore how this would be integrated 
in the tooling and in gnome-builder.

How it works:
For each app that you want to build, you create a manifest.json 
(example: https://people.gnome.org/~gcampagna/manifest.json ) file and 
a ostbuild folder in which you build.
You run
ostbuild make resolve -n
ostbuild make xdg-app -n

The first step is the familiar resolve step, it will download all git 
repositories, download any patch and produce a snapshot.json.
The second step will create a xdg-app workdir and initialize the 
xdg-app build in it. Then for each component it will construct a 
buildroot as xdg-app/files (which xdg-app mounts as /app during the 
build), checkout the sources, build the component, extract the 
artifacts from the build as /runtime /devel /docs /debug trees, commit 
the trees in the ostree repo. Finally, in the last step it runs a 
compose by checking out all component trees, running triggers and 
committing again. Additionally, when composing the runtime tree, it 
calls out to xdg-app to finalize the build directory, which actually 
builds the xdg-app (writes the metadata, extracts the exports).
The result is an xdg-app stored in the local ostree repository, which 
can be installed by adding the repo as a remote for the user repo, 
pulling and installing.

The details of xdg-app builds are, in order:
xdg-app build-init: prepares metadata in app build dir
xdg-app build: enters the sandbox and runs a command in it - this is 
what we use to actually build each component with the right paths and 
prefixes
xdg-app build-finish: finalizes the build, writes metadata, extracts 
files
xdg-app build-export: commits the built app to the repo and signs it
xdg-app build-bundle: extracts the xdg-app commit and makes a bundle 
(based on a ostree static delta)

The system relies on the runtimes and sdks being installed in some 
place that xdg-app knows of - ie, you must "xdg-app install-runtime 
org.gnome.Sdk 3.18" beforehand. I only deal with the app components 
here.

Hope this helps understanding the work, and cheers,

Giovanni