Re: [BuildStream] Questiont: How to handle monorepos in buildstream

[William Salmon <will salmon codethink co uk>]

On 18/01/2019 18:27, Yates, Matthew (StoreOnce) wrote:
> Apologies if this isn't the correct forum for this type of discussion.

I think this is a pretty good place for this sort of discussion, the 
Mailing list tends to get a lot of developer traffic but it is also 
perfect for this sort of discussion.

I think there are going to be a number of ways of getting this to work. 
This is my point of view.

> [snip]
> 2. Have a separate element for each build artefact which each has its own ref to the monorepo
>
>    +---------------------+   +---------------+
>    |                     |   |               |
>    | protobuf-python.bst <---+ service-a.bst |
>    |                     |   |               |
>    +---------------------+   +---------------+
>
>    +-------------------+     +---------------+
>    |                   |     |               |
>    | protobuf-ruby.bst <-----+ service-b.bst |
>    |                   |     |               |
>    +-------------------+     +---------------+
>
>      This solves the two problems above, but I have to make constant efforts or create some tooling to make 
> sure all the refs remain in sync.

I think this is the approach I would take. I know its a pain to check 
the ref's but bst-track is meant to be there to help with this, so long 
as both elements are set to track the same branch then, bst-track should 
do all the hard work, of keeping them consistent. and bst-show should 
help with verifying this.

I also would want some tooling in my CI system to make sure that both 
ref's were indeed consistent. I think most people will want to have 
there own custom tooling but I think that if it isnt much more than a 
thin rapper round bst-show then there is something wrong with bst-show 
and we need to put in a Issue for that.

Also, option 2 gives you a bit more flexibility. Like you said, for most 
cases is very important to keep the source ref consistent between 
protobuf-python.bst and protobuf-ruby.bst. But every now and again its 
handy to be able to do something funny for development purposes, and the 
other ways make that much more complex.

Will


PS. Personally I really like options 3 however I think removing the ruff 
edges with be a big challenge.

Despite really liking option3 I think I would still use option 2 however 
but i think buildstream should have the flexibility to do both.