Re: GNOME Modulesets migrating to BuildStream

[Tristan Van Berkom <tristan vanberkom codethink co uk>]

Hi Christian,

When shaking the tree a bit more publicly, something was bound to fall
out, lets figure this out.

On my side, I can say that there is still a ton of work that needs to
be done which can only be delivered after a hard switch to BuildStream
for the GNOME modulesets - delaying this by a whopping additional six
months is a serious setback to the project, which currently has
momentum and as such; viability.


On Thu, 2017-11-09 at 13:44 -0800, Christian Hergert wrote:
> On 11/09/2017 03:20 AM, Tristan Van Berkom wrote:
[...]
> However, many of us plan our cycles before the cycle starts so that we 
> ensure we have time to implement features without burning out. We're 
> already two months into the 3.28 cycle and not far away from the 
> beginning of freezes. Doubly so when you take into account winter holidays.

This is a delicate dance, we made this decision with the release team
at GUADEC actually, to start using BuildStream only after the imminent
stable release was out the door.

At GUADEC, had I considered that you would want to have integration in
Builder before a switch (I clearly did not suspect this at all) I
should have notified you personally at the time.

I still dont think that the timing for a wider public announcement
would have been right at that time; things need to be near perfect
before we start encouraging JHBuild users to use BuildStream instead to
build GNOME.

> This change will undoubtedly affect those of us working on tooling, 
> newcomer documentation, and worflow. So I'm somewhat concerned that I'm 
> having, what appears to be, a large amount of work dropped on my plate 
> mid-cycle if Builder is to not degrade in usefulness for GNOME developers.
>
> How can we ensure that this change happens without breaking Builder 
> users using the jhbuild runtime?

I'm sorry that out of the many things I have to juggle, I had never
considered Builder compatibility to be a blocker for the GNOME release
team to start maintaining GNOME builds in a new format, this does
strike me as an unreasonable requirement.

Frankly looking back, as someone who pushed a lot for the creation of
GtkBuilder - I could not afford to consider Glade support for the new
format as a blocker to a GTK+ release with GtkBuilder, this is not
_exactly_ apples for apples, but it's fairly close I think - it's not
always realistic to expect that all tooling evolves in lock step.

After some thought, here is my suggestion:

 o Discontinue usage of JHBuild modulesets in GNOME releases
   around January as planned, switch the upstream modulesets to
   BuildStream.

   Let's not jeopardize the momentum we have, lets keep the full time
   resources we actually have allocated to the project, working on
   improving things.

 o Keep the JHBuild modulesets on "life support" purely for the sake of
   Builder needs until such a time that Builder grows the features
   it needs to interact with BuildStream.

   By this I mean, that any changes effected to jhbuild modulesets
   are in fact backports of patches to the upstream BuildStream based
   GNOME project - and are done specifically for the sake of GNOME
   Builder users as a stop gap until Builder can grow the feature.

Looking at the jhbuild commit history, this looks like a relatively low
effort affair, especially if it only really needs to be done on an on
demand basis for the vector of Builder users who actually use the
JHBuild integration features. I suspect that as far as Builder using
Apps go, if they are not already using Flatpak, they are going to start
soon.

Also, if you worry about the quality of an interim life support branch
or repo to "carry you over", I would raise that with how ad-hoc JHBuild
is *currently* maintained, I cannot imagine it being worse (gvfs has
been broken for me for an entire week without anyone else even
noticing, recipes was broken with the updated meson, simply because
users are not guaranteed to actually be using the versions of
dependencies which are declared in the modulesets; it's just a mess).

[...]
> Change of this magnitude is a lot of work and we all value the amount of 
> effort you're putting into this.

Thank you for the kind words.

I will in turn make an effort to give you more of a heads up on what is
coming up, because these things are coming soon, and might very well
effect Builder:

  o Starting now, I am working on deprecating the org.gnome.Sdk JSON
    entirely, my hope would be to have it working almost immediately
    when we do the switch in January, so that we could also release
    the 3.28 GNOME SDK from the same upstream modulesets, and at least
    put this problem behind us right away.

    It is realistically possible that we miss that mark for some
    reason though, and that the migration of the GNOME Flatpak SDK
    get postponed.

  o The org.freedesktop.Sdk runtime and org.freedesktop.BaseSdk
    runtimes are in the process of being migrated and will be generated
    from BuildStream build metadata for the 1.8 releases, this is the
    plan, and is being taken care of simultaneously.

    For this, we've created a separate project; because there are more
    interested parties in this runtime build than only Flatpak,
    currently all we have going in terms of infra is:

     - #freedesktop-sdk channel on freenode
     - f.d.o mailing list: https://lists.freedesktop.org/mailman/listinfo/freedesktop-sdk
     - git repo: https://gitlab.com/freedesktop-sdk

    This repo is intended to be migrated to a freedesktop gitlab
    instance once that exists.

  o Bootable images of GNOME builds at least for Intel architectures

    This will inevitably also depend on the freedesktop-sdk project
    to provide the base runtime, which the GNOME modulesets will depend
    on to produce a bootable image (_and_ to produce the GNOME SDK).

    I suppose that for Builder, it might be interesting for testing
    system component "Apps" (like gnome control center or clocks or
    such things); such that you are testing your software against what
    will really be the next generation GNOME environment at any time.

    Maybe you will be interested in being able to automatically
    launch a VM after a build of a given module.

    Eventually, it will also be interesting to roll out bootable VMs
    containing pre-installed Flatpaks, for the sake of testing your
    new GNOME Flatpak and seeing how it integrates against the next
    GNOME release.


Cheers,
    -Tristan