|
|
1
|
+.. _architecture-overview:
|
|
|
2
|
+
|
|
|
3
|
+Remote execution overview
|
|
|
4
|
+=========================
|
|
|
5
|
+
|
|
|
6
|
+Remote execution aims at speeding up a build process and rely on two separate
|
|
|
7
|
+but related concepts that are remote caching and remote execution itself. Remote
|
|
|
8
|
+caching is sharing build outputs while remote execution is handing a build
|
|
|
9
|
+operation to a distant service that is likely to be a powerful build farm.
|
|
|
10
|
+
|
|
|
11
|
+The `Remote Execution API`_ (REAPI) describes a `gRPC`_ + `protocol-buffers`_
|
|
|
12
|
+interface that has three main services for remote caching and execution:
|
|
|
13
|
+
|
|
|
14
|
+- A ``ContentAddressableStorage`` (CAS) service: a remote storage end-point
|
|
|
15
|
+ where content is addressed by digests, a digest being a pair of the hash and
|
|
|
16
|
+ size of the data stored or retrieved.
|
|
|
17
|
+- An ``ActionCache`` (AC) service: a mapping between build actions already
|
|
|
18
|
+ performed and their corresponding resulting artifact.
|
|
|
19
|
+- An ``Execution`` service: the main end-point allowing one to request build
|
|
|
20
|
+ job to be perform against the build farm.
|
|
|
21
|
+
|
|
|
22
|
+The `Remote Worker API`_ (RWAPI) describes another `gRPC`_ + `protocol-buffers`_
|
|
|
23
|
+interface that allows a central `Bots` service to manage a farm of workers.
|
|
|
24
|
+
|
|
|
25
|
+BuildGrid is combining these two interfaces in order to provide a complete
|
|
|
26
|
+remote caching and execution service. It's high level architecture can be
|
|
|
27
|
+represented like this:
|
|
|
28
|
+
|
|
|
29
|
+.. graphviz::
|
|
|
30
|
+ :align: center
|
|
|
31
|
+
|
|
|
32
|
+ digraph remote_execution_overview {
|
|
|
33
|
+ node [shape = record];
|
|
|
34
|
+ ranksep = 0.7
|
|
|
35
|
+
|
|
|
36
|
+ client [label = "Client"];
|
|
|
37
|
+ controller [label = "Controller"];
|
|
|
38
|
+
|
|
|
39
|
+ subgraph cluster_worker0 {
|
|
|
40
|
+ label = "Worker 1";
|
|
|
41
|
+ labeljust = "l";
|
|
|
42
|
+ bot0 [label = "{Bot|Host-tools\n\n}"];
|
|
|
43
|
+ }
|
|
|
44
|
+
|
|
|
45
|
+ subgraph cluster_worker1 {
|
|
|
46
|
+ label = "Worker 2";
|
|
|
47
|
+ labeljust = "r";
|
|
|
48
|
+ bot1 [label = "{Bot|BuildBox\n\n}"];
|
|
|
49
|
+ }
|
|
|
50
|
+
|
|
|
51
|
+ client -> controller [dir = "both",
|
|
|
52
|
+ headlabel = "REAPI",
|
|
|
53
|
+ labelangle = -70.0,
|
|
|
54
|
+ labeldistance = 2.4,
|
|
|
55
|
+ labelfontsize = 10.0];
|
|
|
56
|
+ controller -> bot0 [dir = "both"];
|
|
|
57
|
+ controller -> bot1 [dir = "both",
|
|
|
58
|
+ labelangle = 40.0,
|
|
|
59
|
+ labeldistance = 3.0,
|
|
|
60
|
+ labelfontsize = 10.0,
|
|
|
61
|
+ taillabel = "RWAPI"];
|
|
|
62
|
+ }
|
|
|
63
|
+
|
|
|
64
|
+.. _Remote Execution API: https://github.com/bazelbuild/remote-apis/blob/master/build/bazel/remote/execution/v2
|
|
|
65
|
+.. _gRPC: https://grpc.io
|
|
|
66
|
+.. _protocol-buffers: https://developers.google.com/protocol-buffers
|
|
|
67
|
+.. _Remote Worker API: https://github.com/googleapis/googleapis/tree/master/google/devtools/remoteworkers/v1test2
|
