Re: Recursive pipelines and subprojects

[Tristan Van Berkom <tristan vanberkom codethink co uk>]

On Tue, 2017-11-14 at 11:29 +0100, Jürg Billeter wrote:
> Hi all,
>
> I've been working on the recursive pipeline feature to support
> BuildStream projects that depend on other BuildStream projects in a
> practical and convenient manner. There has been some discussion on IRC
> but not yet on the mailing list.

To provide just a little more background, this project has been called
"recursive pipelines" because it comes directly from the initial design
drafts of BuildStream from over one year ago.

An outline of how I had originally envisioned the feature can be found
on the roadmap:

    https://wiki.gnome.org/Projects/BuildStream/Roadmap/RecursivePipeline

As such, I would say that this carries a *lot* more weight than mere
IRC discussion as hinted above, and it predates the existence of the
mailing list by a long shot.

This feature in a nutshell, is our approach to the problem of having a
single tree full of build metadata for building everything under the
sun (I.e. consider buildroot or yocto. In fairness yocto does has a
kinda sorta solution to this with layers, but in practice these are
more often used to override some bits, and I feel that it still falls
short of proper modularity in the build stack).

As I have discussed this at length, quite frequently over the past year
with Jürg, and since reading this pretty much matches up perfectly with
what we discussed last time we brainstormed this, I have no problems
with the proposal whatsoever.

I do agree that conflict checks based on project name *seems* a bit
weak, but I don't see a better way to do this at the moment. The source
URL is completely inappropriate, because in general Sources should be
using aliases, and the backing URL is *intended* to be allowed to
change, as a provision for eventual features for source tree mirroring.

Best Regards,
    -Tristan