Hi,
I've been following both BuildStream and Bazel, and am seeing some potential overlaps where we could look into leveraging the collective experience and brain power, rather than solve it individually.
What is Build Farm?
-------------------
A variant of this code base is in use at Uber, and is seeing interest and contributions from other parties.
The services as described in:
are in summary:
- Content Addressable Storage: responsible for storing all Inputs and Outputs indexed by content hash.
- Execution Service: responsible for executing Actions on Inputs on Remote Workers, or reusing Action Results from the Action Cache Service.
- Remote Worker: responsible to execute Actions on Inputs locally, at the direction of the Execution Service. Also responsible for producing Outputs, storing them in Content Addressable Storage. Lastly returning an Action Result.
- Action Cache Service: responsible for mapping Actions to Action Results.
A client is responsible for making sure all inputs exist in Content Addressable Storage, including the command to execute. The client then contacts the Execution Service with an Action to execute the command. It gets back an ActionResult.
For a decent understanding, I would consider the Design Document: Remote Execution API required reading.
BuildStream Build Farm Client
-----------------------------
When thinking about BuildStream, I see potential for using the Remote Execution API for all actual builds, when it comes to a distributed setup. This would work by having all sources be uploaded into Content Addressable Storage, composing a tree that contains all dependencies (essentially a representation of the staging area). Then contacting the Execution Service with the command(s) that are specified in bst YAML. Then process the Result; use it as an Input to a reverse dependency element build.
The Remote Worker is solely responsible for staging, sandboxing, execution and storing resulting artifacts. This would need to run on the actual target platforms (unless you are cross compiling).
With Content Addressable Storage in the mix, the role of the ostree based artifact cache might become redundant. Or in terms of implementation, an alternative, just like the tar cache.
In terms of dependencies it looks like adding grpc, which doesn't look too bad.
Wait But Why?
-------------
I think that if BuildStream is used in a context with decent churn on codebases there is a desire to parallelize builds as much as possible.
In organizations with significant amounts of existing code bases and build systems, as well as having significant churn, I can see both BuildStream and Bazel having a potential to be adopted and co-exist for a decent amount of time, if not indefinite.
BuildStream, being a meta-build system, allows for bridging between existing build systems and Bazel. The idea being that in Bazel you could consume the binary artifacts produced by BuildStream builds (
https://docs.bazel.build/versions/master/external.html#non-bazel-projects). These binaries would still be fully rebuildable, and have provenance, as built by BuildStream.
I imagine a way to make dependencies from Bazel available to BuildStream as well. Maybe through a source plugin to fetch from Content Addressable Storage, like the ostree source plugin, combined with the import element.
The BuildStream dependency resolution would prevent from crossing the streams.
BuildStream will cater for all projects where it is, either prohibitive, or the ROI is relatively low, to convert to Bazel. One example being autotools based projects released by third parties.
I then see two build systems that are operating at a different level, namely meta-build system and build system, that have a fairly similar architecture for executing [build] commands. Both operate by creating a sandbox, staging dependencies, and executing commands in that sandbox to produce outputs. To me it makes sense to investigate whether there is room to share infrastructure.
That will allow for developing, maintaining, operating and scaling a single set of underlying services, benefitting both systems. To enable that we would need to look at adopting common APIs.
I guess now it's time to don those fireproof pants... :) Thoughts?
Cheers,
Sander
