Re: [BuildStream] BuildStream releasing model

[Tristan Van Berkom <tristan vanberkom codethink co uk>]

On Tue, 2018-07-31 at 15:02 +0200, Paul Sherwood wrote:
> ---
>
>
>
> On 2018-07-31 12:05, Tristan Van Berkom wrote:
> > > Please let's not use 'stable' since what engineers typically mean by 
> > > 'stable' is often not what users expect or need.
> >
> > I leave Agustin's answer as it is a good one, but note that this is
> > only a matter of nomanclature.
>
> No, it's not only a matter of nomenclature. It's often a sign of a 
> fundamental misunderstanding.
>
> Developers create 'stable branch' thinking that users want a branch 
> without the risky/fast rate of change they have in their development 
> flow.
>
> Wrong. Generally users want a release that works, and once they have it 
> won't upgrade by default. Unless the upgrades are unavoidable (something 
> is broken), infrequent and and/or automated, users won't consume them.

I think you presume a lot about the developers motivations for creating
a stable branch.
 
The main reason we use stable branches (IMO), is to allow us to address
issues in the field, with near certainty that we are not introducing
new regressions at the same time.

> Users would prefer not to think about upstream's branching at all, until 
> something goes wrong.
>
> This is true for users who are developers, just as much as for 
> non-technical users.

Unless they are developers of BuildStream itself, or are volunteering
to test an unvetted development line; Users should really not be using
using git to install BuildStream, and should not need to understand
anything about the dependencies, or the branching scheme used to
develop the software that ends up on their computer, usually via a
package manager in the distro they use, or self installing binary
published by the upstream.

The alternative is pip, although BuildStream is not very suitable for
releasing through pip, because it will always need some non-Python
dependencies and integration with the host.

The fact that the project is still barely growing out of it's infancy,
and is not widely available on distros, is not an excuse to change our
branching scheme and development model.

It is unfortunate that the regular user needs to know everything that a
distro packager would need to know in order to install BuildStream,
while I don't think that knowledge is currently a lot, the user should
not need to know any of it - still this is just another growing pain
and will pass.

> And the problem is compounded when the users are layered in downstream 
> projects, which is one of the expected usecases of buildstream.
>
> I first got involved in this topic regarding the 'long term stable' 
> LTS/LTSI kernel approach. It just **doesn't work** in practice. GKH used 
> to say "users of this patch series must upgrade" every couple of weeks. 
> But virtually none of the downstream projects was doing so afaict. And 
> in commercial projects, with vendor passing kernel on to other vendor, 
> who bundles it to final customer, there is never any possibility of it 
> happening.
>
> > > Please can someone spell out **the reasoning** for this odd/even 
> > > model, 
> > > without digressing to history or references from other projects.
> > >
> > > My counter-proposal is:
> > >
> > > 1) we tag a release from time to time (frequency tbd... let's guess 
> > > at 
> > > quarterly). new users are told to use the latest tag
> > > 2) after the release we merge some new functionality, test it, fix 
> > > the 
> > > breakages, and ensure that we are backwards compatible with content 
> > > from 
> > > our previous releases
> > > 3) we tag a new release
> > > 4) we tell existing users to migrate forward to the latest release 
> > > within a time window (tbd, let's say 1 year)
> > > 5) if we identify crash/security bugs we make them in master then 
> > > cherry-pick/backport to old supported tags, or state that we have to 
> > > EOL 
> > > old tags early
> > >
> > > I'm suggesting this to emphasise that I think we should always have 
> > > one 
> > > release live, and one that we are working towards, and everything 
> > > else 
> > > is secondary.
> >
> > This is basically exactly what we have, except we have it with branches
> > and not only with tags.
>
> As I've said, I believe branches are the wrong approach for users.
>
> > Now more than every before (and hopefully more than ever in the
> > future), it is of the utmost importance to give the master/development
> > branch some leeway for development.
>
> I'm fine with that. This discussion is about release model, not 
> development model.
>
> > At the current rate that contributors want to develop features, it is:
> >
> >   A.) Impossible to safely push releases from master onto users who
> >       need things to "just work" or at the very least not regress.
>
> Agreed. Again, that's not what I'm suggesting.
>
> >       Things need time to materialize in master, and branching early
> >       is a move we did especially to cater to the contributors who
> >       need more velocity in their development work.
>
> Not relevant to the releasing model?
>
> >   B.) Also important to provide development snapshot releases from
> >       the development line, for those users who willingly take the
> >       risk, play with things early, and help us identify issues
> >       before landing a new 1.4 that is reasonable usable and does
> >       not regress compared to 1.2.
>
> Those are 'early adopter' users, I think.
>
> I had thought, and still hope, that bst is aiming to be suitable for 
> construction and maintenance of super-longterm critical infrastructure 
> projects.

At this stage I am a bit confused.

You are proposing changes to our release model, and how we internally
manage our version control system, but it seems your are crossing this
internal development process with how users obtain BuildStream - while
currently users have to manually install from git, that in itself is a
bug which will be addressed.

Since this in itself needs to be fixed, we should not design for an
unideal, temporarily broken situation where our distribution channel is
still a manual installation directly from git - this separation is
important, for both upstream and for users.

Before continuing, I think:

  * Your proposal needs a clear problem statement.

    People on this list need to have an understanding and agreement
    that a given thing is a problem, before we can even begin to
    analyze whether your proposed change actually addresses the problem
    it purports to solve, does not regress other necessary aspects of
    our processes, and is generally agreed to be the most elegant
    solution.

  * In your proposed changes, you need to at least fix point (5), as it
    is logistically impossible to backport a bugfix to a git tag.

  * Keep in mind that what is in master now, will not be usable for
    months, this is the scale of amassed technical debt (which was
    altogether justified and necessary in this instance in order to
    improve velocity of development).

    Users using master is just not going to happen in this climate, but
    I have hope that we will have willing testers later in the cycle.

It is quite possible that after understanding the problems you
perceive, someone here will be able to come up with a solution for it
with a much less disruptive change than the one you have in mind, too.

Cheers,
    -Tristan