Re: component groups and preparatory patches for sdk/platform work

[Alexander Larsson <alexl redhat com>]

(Cross-posting this to gnome-os-list as it is related)

On tor, 2014-10-30 at 18:31 -0400, Colin Walters wrote:
> On Wed, Oct 29, 2014, at 04:21 AM, Alexander Larsson wrote:
> > I posted my patches here:
>
> Can you briefly describe the expected end result here?  So for example
> are you proposing that people would download the SDK (and apps?)
> from build.gnome.org?
>
> And are you thinking that the tools would be packaged for downstream distributions?

So, I've been thinking a bit about how this should work, and I'm not
sure i'm 100% sold on using gnome-continuos for the platform stuff. I
like some parts of it, but not others.

One thing I do like is the yocto base. It gives us a "supported, but by
someone else" base that is pretty modern and easy to build. Using the
-devel image from that in a container like i do with gnome-sdk is very
useful and cool. I think this is something we should use (although i
need to rebase it when 1.7 final is out).

One thing I don't like is all the complexity involved in making it build
incrementally. The updating of git mirrors, the resolving for changes,
partial builds, etc. For the usecase we have here, which is to build
release-quality binaries, the incremental work is not what we want as it
gives you no repeatability. So, all this complexity is not really buying
us anything here. I would rather we did a from-scratch build of the
entire gnome base image for every release.

Another thing continuous does is building and installing the non-base
modules and extracts runtime/devel/docs/debuginfo subtrees which it can
then (via ostree) combine into a build. I'm a bit undecided about this
part. One option is to extract this part and use it for a more
streamlined "build a set of modules in order" operation. This is quite
doable for the basic platform and sdk relasese, but I would also like
the tools used for building the sdk/platform to be useful to people
building applications, for instance as a way to build pre-build
dependencies that app developers can easily bundle. 

In this usecase I think the ostree subtree approach would suffer
compared to something a bit more featureful like existing package
managers. For instance, once you get out of the simple linear "this set
of module make up the platform" you really need dependency and version
tracking to avoid e.g. forgetting to bundle a dependency.

One could perhaps incrementally add features to gnome-continuous (or
some related project based on this) to make it more useful here, but
this will  make g-c more complicated for the current, very important
task of doing continuous integration builds for gnome, and be
"duplicated" work with what gnome-sdk (isolated building) and existing
package managers (packaging, dependency tracking) do.

So, I want to experiment with a different path. We use the g-c yocto
base, with the extra work I've done to make it build a minimal
non-bootable platform image. Then we use gnome-sdk to build stuff inside
a container with this base, exactly like how you build apps. To drive
the builds we use rpm[1], producing packages that we can then install
when building apps or runtimes. The packages that goes into the runtime
is build with prefix /usr, but we can also build packages in the
app-bundling prefix (/self atm) for easy bundling into apps.

Then we can use the base images + rpms to create a final image trees for
the runtime/sdk and import this into ostree, which we can use to push
images to users. We probably also have to make some kind of fake rpm for
the base system so that dependencies on it from bundled rpms are
handled.

Does this seem reasonable?

I'll go play with it to see if it works.

[1] Technically it could be dpkg or whatever, but rpm is what i know
best and its good enough for our use.