Re: component groups and preparatory patches for sdk/platform work

[Colin Walters <walters verbum org>]

On Thu, Nov 6, 2014, at 10:22 AM, Alexander Larsson wrote:

> So, I've been thinking a bit about how this should work, and I'm not
> sure i'm 100% sold on using gnome-continuos for the platform stuff. I
> like some parts of it, but not others.

Yeah.

> One thing I don't like is all the complexity involved in making it build
> incrementally. The updating of git mirrors, the resolving for changes,
> partial builds, etc. For the usecase we have here, which is to build
> release-quality binaries, the incremental work is not what we want as it
> gives you no repeatability.

1) Anytime we want to rebuild everything from source, it's just a matter of setting buildAll=true and bumping 
the manifest epoch.  
2) It would make sense, and not be too terribly hard to have a separate builder that does this at some 
interval (once a week?)

> Another thing continuous does is building and installing the non-base
> modules and extracts runtime/devel/docs/debuginfo subtrees which it can
> then (via ostree) combine into a build. I'm a bit undecided about this
> part. One option is to extract this part and use it for a more
> streamlined "build a set of modules in order" operation. This is quite
> doable for the basic platform and sdk relasese, but I would also like
> the tools used for building the sdk/platform to be useful to people
> building applications, for instance as a way to build pre-build
> dependencies that app developers can easily bundle. 

Right...though one would also need to support 3rd parties using their
own custom build systems.

> In this usecase I think the ostree subtree approach would suffer
> compared to something a bit more featureful like existing package
> managers. 

I don't disagree.  But on the other hand I'm pretty proud of the total
lack of dependency resolution in the top half of Continuous ;)

> For instance, once you get out of the simple linear "this set
> of module make up the platform" you really need dependency and version
> tracking to avoid e.g. forgetting to bundle a dependency.

Wouldn't your app just fail to run, and that would be obvious?

> So, I want to experiment with a different path. We use the g-c yocto
> base, with the extra work I've done to make it build a minimal
> non-bootable platform image. Then we use gnome-sdk to build stuff inside
> a container with this base, exactly like how you build apps. To drive
> the builds we use rpm[1], producing packages that we can then install
> when building apps or runtimes. The packages that goes into the runtime
> is build with prefix /usr, but we can also build packages in the
> app-bundling prefix (/self atm) for easy bundling into apps.

Mmmm....do you see us taking apps out of the Continuous manifest and
using this system on build.gnome.org?  Or we keep Continuous as is
and this is a separate system?

Or a hybrid where core apps stay in the manifest and other apps use this?

I'd really like to have at least some core applications continuously delivered
and *tested*.  We've barely tapped this now, but I personally think
it's really quite cool that we automatically at least run+screenshot Epiphany when GTK+
changes (or for that matter, glib, kernel).  I'm not aware of anyone else that comes
close to the speed of re-integration here.

> Then we can use the base images + rpms to create a final image trees for
> the runtime/sdk and import this into ostree, which we can use to push
> images to users. We probably also have to make some kind of fake rpm for
> the base system so that dependencies on it from bundled rpms are
> handled.

Also put the whole thing in a Docker container!

> [1] Technically it could be dpkg or whatever, but rpm is what i know
> best and its good enough for our use.

I suspect you'll end up with some sort of spec generation/package abstraction like OpenEmbedded does.  I feel 
like anything else is far, far too tedious to maintain.