|
|
1
|
+.. _architecture-overview:
|
|
|
2
|
+
|
|
|
3
|
+Remote execution overview
|
|
|
4
|
+=========================
|
|
|
5
|
+
|
|
|
6
|
+Remote execution aims at speeding up a build process and rely on two seperate
|
|
|
7
|
+but related concept that are remote caching and remote execution itself. Remote
|
|
|
8
|
+caching is sharing build outputs, renote execution is handing a build operation
|
|
|
9
|
+to a distant service that is likely to be a powerful build farm.
|
|
|
10
|
+
|
|
|
11
|
+The `Remote Execution API`_ (REAPI) describes a `gRPC`_ + `protocol-buffers`_
|
|
|
12
|
+interface that has three main services implementing remote caching and
|
|
|
13
|
+execution:
|
|
|
14
|
+
|
|
|
15
|
+- A ``ContentAddressableStorage`` (CAS) service: a remote storage end-point
|
|
|
16
|
+ where content is addressed by digests, a digest being a pair of the hash and
|
|
|
17
|
+ size of the data stored or retrieved.
|
|
|
18
|
+- An ``ActionCache`` (AC) service: a mapping between build actions already
|
|
|
19
|
+ performed and their corresponding resulting artifact.
|
|
|
20
|
+- An ``Execution`` service: the main end-point allowing one to request build
|
|
|
21
|
+ job to be perform against the build farm.
|
|
|
22
|
+
|
|
|
23
|
+The `Remote Worker API`_ (RWAPI) describes a `gRPC`_ + `protocol-buffers`_
|
|
|
24
|
+interface that let a central bot service managing a farm of workers.
|
|
|
25
|
+
|
|
|
26
|
+BuildGrid use both of this interface in order to provide a complete remote
|
|
|
27
|
+caching and execution service. It's high level architecture can be represented
|
|
|
28
|
+like this:
|
|
|
29
|
+
|
|
|
30
|
+.. graphviz::
|
|
|
31
|
+
|
|
|
32
|
+ digraph remote_execution_overview {
|
|
|
33
|
+ node [shape=record];
|
|
|
34
|
+ nodesep=1.5;
|
|
|
35
|
+
|
|
|
36
|
+ client [label="Client"];
|
|
|
37
|
+ controller [label="Controller"];
|
|
|
38
|
+
|
|
|
39
|
+ client -> controller [dir="both",labelfontsize=10,labeldistance=2.4,labelangle=-70,headlabel="REAPI"];
|
|
|
40
|
+
|
|
|
41
|
+ subgraph cluster_worker0 {
|
|
|
42
|
+ label="Worker 1";
|
|
|
43
|
+ labeljust="l";
|
|
|
44
|
+ bot0 [label="{Bot|Host-tools\n\n\n}"];
|
|
|
45
|
+ }
|
|
|
46
|
+
|
|
|
47
|
+ subgraph cluster_worker1 {
|
|
|
48
|
+ label="Worker 2";
|
|
|
49
|
+ labeljust="r";
|
|
|
50
|
+ bot1 [label="{Bot|BuildBox\n\n\n}"];
|
|
|
51
|
+ }
|
|
|
52
|
+
|
|
|
53
|
+ controller -> bot0 [dir="both"];
|
|
|
54
|
+ controller -> bot1 [dir="both",labelfontsize=10,labeldistance=2.4,labelangle=-90,taillabel="RWAPI"];
|
|
|
55
|
+ }
|
|
|
56
|
+
|
|
|
57
|
+.. _Remote Execution API: https://github.com/bazelbuild/remote-apis/blob/master/build/bazel/remote/execution/v2
|
|
|
58
|
+.. _gRPC: https://grpc.io
|
|
|
59
|
+.. _protocol-buffers: https://developers.google.com/protocol-buffers
|
|
|
60
|
+.. _Remote Worker API: https://github.com/googleapis/googleapis/tree/master/google/devtools/remoteworkers/v1test2
|
