BuildStream Embarks on Remote Execution

[finn <finn ball codethink co uk>]

Hi,

Here at BuildStream, we are developing a Free Software tool for building and integrating software stacks. Buildstream is designed to work with a set of YAML files which describe the targets and dependencies, evaluates the various build instructions and deploys the resulting targets. It supports multiple build-systems and can output in a range of formats.

We are interested in creating a remote execution service by implementing the Remote Execution API (REAPI) and the Remote Workers API (RWAPI), taking inspiration from the Bazel and Uber implementations. Buildstream's sandboxing and action model differs from Bazel in that the actions (e.g `make` of a third party module) are executed in a sandbox that doesn't have access to host tools or libraries. The planned implementation will be designed to work well with this and will not be designed to be BuildStream specific.

A rough plan is in the pipeline which shows an initial outline to be implemented in Python with the design to be expanded on in the future. The first focus will be a simple implementation of the REAPI. This will register workers, consult the action cache and queue actions ready for workers to request. The details of the implementation of the RWAPI are still being worked on.

Currently though, we are blocked by the name. We want a name distinct from BuildFarm, here are some notable suggestions thus far:

[*] Conflux
[*] Conduit
[*] BuildGrid
[*] Faucet

Please reply if you have any comments, suggestions, need clarification of any of the points or have any ideas for what to call this implementation.

Thanks,
Finn