Re: [BuildStream] Summary of the 'Staging multiple elements in bst shell' discussion at the gathering

[James Ennis <james ennis codethink com>]

Hi Tristan,

Thanks for taking the time to reply.

On 2019-02-05 16:34, Tristan Van Berkom wrote:
> Hi James,
>
> I was not present at this discussion, thank you for summarizing it in
> detail !

No problem!

[...]

> On Mon, 2019-02-04 at 16:36 +0000, James Ennis via BuildStream-list 
> wrote:
> > Hi all,
> [...]
> > # Summary of discussion at the gathering
> > First of all, we discussed what the desired and viable use-cases were 
> > here. It was agreed that for now, it makes sense to be able to stage 
> > multiple runtime elements in the shell so that we could, for example, 
> > include our favourite debugger/text editor. However, regarding the 
> > question of staging multiple elements into the build shell - it 
> > doesn't 
> > make sense to have multiple build environments in the same shell.
>
> There are runtime dependencies and build time dependencies, there is no
> such thing as "runtime elements" and there *is* such a thing as "build
> elements".

Apologies if this was unclear. I think this would be better phrased:
"it makes sense to be able to stage multiple elements as extra runtime
dependencies"

> When we say, that staging multiple elements into a build shell doesnt
> make sense because it doesnt make sense to have multiple build
> environments... maybe you mean that it does not make sense to stage two
> separate elements for building ? (i.e. with their sources staged) ?

Yes.

> Does this somehow remove the ability of staging additional
> complementary elements in the context of invoking a build shell for a
> given element ?

Yes, and as mentioned above, this is the element for which we stage the
sources. However, for the additional elements, which will be staged as
extra runtime dependencies, it was discussed at the gathering that we
would stage these separately by 'merging the extra runtime tree'.

> I see that the element you are building is the one which should
> determine the environment, so there should always be one element
> hardwired to be "the build element" when launching a build shell so
> that this is always clear, but I don't think this effects our ability
> to stage more artifacts.
>
> Further, we should recognize that this is not specific to build shells,
> the environment variables specified by a given target element will also
> be the ones used for a regular `bst shell` which you might use to
> launch and test a graphical application on your laptop.

Ahh, this was not really discussed at the gathering. However, for a 
'normal' shell, the current implementation (in !909 [0]) is to merge 
environment variables from all specified elements. Which may not be the 
best idea due to possible conflicts. Perhaps there needs to be further 
discussion as to what the right behaviour is here.

To give my opinion on the issue, regarding the build shell, we're in 
agreement that the build shell can only specify one target element, and 
from what we've discussed, it does not look like we want to merge any of 
the 'extra' environment variables from the additional runtimes here.
So perhaps, for the runtime shell, we should try to be consistent and go 
for a main target element, of which, we only merge its environment 
variables.

[...]

> I wonder if you have considered staging order in all of this (incase
> you want to explicitly prioritize conflicting dependency trees for the
> overlaps), and if you have consider the `--directory` options discussed
> before.
>
> I.e. you might want to stage a literal host directory which has a bunch
> of host tools in it which work for the specified arch, and you might
> want to stage it before or after the artifacts, allowing you to get
> your host tools into the shell without creating any elements or
> interacting with the project data.

Regarding staging order, it was agreed that we'd be merging the 
additional dependency tree afterwards.

The use of the `--directory` option was not discussed at this gathering 
but I agree that this could be a useful alternative.

> Cheers,
>     -Tristan

Cheers,
James