Re: [BuildStream] Execution environments

[Jürg Billeter <j bitron ch>]

On Thu, 2018-11-01 at 20:28 +0100, Jürg Billeter wrote:
> Cache key: We already include `os` and `arch` in the cache key
> calculation, currently using the value from the host. However, as the
> existing `arch` is based on `uname -m` instead of the proposed list,
> cache keys will still change.
>
> Open issue: The current `arch` project option type is documented and
> implemented using `uname -m` as default. However, as described above,
> `uname -m` varies across systems and is thus not suitable. Any
> suggestions how to handle this with minimal breakage of existing
> projects?

A possibility to solve this issue is to use the proposed OS-independent 
arch names for `arch` project options only if the project declares the
new format-version.

We don't currently use this approach (change behavior based on project
format-version), however, it might provide a reasonable backward
compatibility story, as long as we don't overuse it. This approach
might also be useful when/if we introduce cache key stability.

Related to this, would it also make sense to report an error when a
project uses a feature that wasn't available in the format version
declared in project.conf? This would force projects to declare the
right format version and provide a better error message for users of
old BuildStream versions. (We should not retroactively do this for past
format versions, but we could do it for future format version bumps).

On the topic of project options and execution environments, I think it
would also make sense to introduce a new project option type `os` that
works similar to the `arch` option type. This would allow multi-
platform projects to default to the host OS instead of requiring their
users to explicitly specify the corresponding project option.

Cheers,
Jürg