Re: [BuildStream] Preserving artifacts out of workspaces

[William Salmon <will salmon codethink co uk>]

Hi Tristan

Thanks for the detailed feedback

> My comment on 512:
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> As far as I can see, the following invariants make this a non-issue:
>
> * The BuildElement uses Sandbox.mark_directory() to declare that
>    build-root and install-root will not be readonly during the
>    Element.assemble() stage.
>
> * We currently do not support out of tree builds where
>    builddir != srcdir
>
>    * We only support builddir != srcdir where the builddir is a
>      subdirectory
>
> * If we want to support completely out of tree builds, this does not
>    need to precipitate such a dramatic change
>
>
> A straight forward way of implementing out-of-tree builds, would be:
>
> * Stage the sources to the source/ subdirectory, using the Source base
>    class understood directory configuration
>
> * Set the command-subdir variable to the source/ directory, so that all
>    commands are run in the subdir where the sources are staged (this
>    will "just work" for any build element).
>
> * Invoke the configure script (or what have you), informing it that the
>    build dir is ../build
>
>    * Now the build dir and source dir are both subdirectories of
>     %{build-root}, a nicely writable location.
>
> * Workspaces will "just work", since both the Source.stage() and
>    Source.init_workspace() methods already take into account the
>    automatic directory configuration which can be set for any Source.

I have tried to recreate what you described in 
https://gitlab.com/BuildStream/buildstream/commit/dab4489569652d8b37ff57b05f1f1e34059114f5

Is this approximately what you meant?

It seems to work ok, although there are a few funnies I am still working 
through.

If i run `bst build hello.bst`

Then it builds fine and the hello program can be run in the shell or via 
`bst shell hello.bst -- hello`

But if I make a workspace first and then run `bst build hello.bst` it 
fails but if I make a build folder in the workspace then `bst build 
hello.bst` runs.

Then I can look in the workspace and see my build artifacts or "Element 
output"(s) in the build directory.

I can also run `bst shell hello.bst --build` and run the "Element output"

> Does this satisfy the use case sufficiently that we can just close this
> and move on ?
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

If my example is what you were trying to describe and all the relevant 
parties have a look and try out my example implementation of your 
suggestion and are happy with this way of working then I propose:

* Given that we have got this far in to disusing this before you pointed 
this out I think it might be a good idea to properly document this, 
maybe a tutorial, and then have the issue closed once the MR adding the 
docs has been merged?

That dose everyone think?

Many Thanks
Will