Re: [BuildStream] BuildGrid and BuildStream 1.4 and API concerns

[Tristan Van Berkom <tristan vanberkom codethink co uk>]

Hi again,

I wanted to add to this in case this in the hope that we don't blow
things out of proportion here, let's not make a mountain out of what
might just be a mole hill.

On Sun, 2018-12-02 at 16:08 +0900, Tristan Van Berkom via BuildStream-list wrote:
[...]
> In this instance it can be easily avoided by *temporarily* bundling
> versions of the BuildGrid service which are known to work for a given
> minor point BuildStream release line - i.e. users have the same
> guarantees that they have with the artifact server, at a fairly low
> cost.
>
> In other words, lets not block the whole feature away from prime time
> just because the REAPI spec is not ready, we really only need to block
> interoperability guarantees until that time.

To clarify, I am also very much against the bundling approach, as it
implies a lot of logistic overhead, just for a situation which will
hopefully only last one release cycle.

My worry about an experimental flag is not only because of the
implications of this, which we've been trying to avoid so far, but also
because of the indefinite time frame that you imply - if this is
experimental without any promise of making a stable API contract at any
definitive time, this is a very bad situation.

It would be much less concerning if the feature was experimental just
because we admit to having missed our targets this year; it's a big
feature and it's just not ready yet, is a much better statement than
one which implies we maintain unstable/experimental codepaths
indefinitely.

There are also other avenues to consider, versioned, parallel
installable instances of BuildGrid would also provide the required
stability. I.e. BuildStream does not necessarily need to have a minimal
bound (optional) dependency on BuildGrid, it just needs to know at all
times that the version of BuildGrid being accessed is a compatible one,
so that it can inform the user which version of BuildGrid is needed in
cases where things are incompatible, instead of just exploding with
cryptic error messages or stack traces.

Versioning of the client facing APIs is not particularly arduous
especially if backwards compatibility is not advertised, this is really
not a lot to ask and gets us out of both of the undesirable
"experimental flag" and "bundled service" territories, which I think
are both a bit overkill just for the sake of avoidance of responsibly
versioning an API.

Cheers,
    -Tristan