[Notes] [Git][BuildGrid/buildgrid][master] 2 commits: docs: Load the graphviz extension for diagrams



Title: GitLab

Martin Blanchard pushed to branch master at BuildGrid / buildgrid

Commits:

4 changed files:

Changes:

  • docs/source/architecture.rst
    1
    +.. _architecture:
    
    2
    +
    
    3
    +Architecture
    
    4
    +============
    
    5
    +
    
    6
    +This section provides details of the overall BuildGrid architecture.
    
    7
    +
    
    8
    +.. toctree::
    
    9
    +   :maxdepth: 3
    
    10
    +
    
    11
    +   architecture_overview.rst
    \ No newline at end of file

  • docs/source/architecture_overview.rst
    1
    +.. _architecture-overview:
    
    2
    +
    
    3
    +Remote execution overview
    
    4
    +=========================
    
    5
    +
    
    6
    +Remote execution aims to speed up a build process and to rely on two separate
    
    7
    +but related concepts that are remote caching and remote execution itself. Remote
    
    8
    +caching allows users to share build outputs while remote execution allows the running
    
    9
    +of operations on a remote cluster of machines which may be more powerful than what
    
    10
    +the user has access to locally.
    
    11
    +
    
    12
    +The `Remote Execution API`_ (REAPI) describes a `gRPC`_ + `protocol-buffers`_
    
    13
    +interface that has three main services for remote caching and execution:
    
    14
    +
    
    15
    +- A ``ContentAddressableStorage`` (CAS) service: a remote storage end-point
    
    16
    +  where content is addressed by digests, a digest being a pair of the hash and
    
    17
    +  size of the data stored or retrieved.
    
    18
    +- An ``ActionCache`` (AC) service: a mapping between build actions already
    
    19
    +  performed and their corresponding resulting artifact.
    
    20
    +- An ``Execution`` service: the main end-point allowing one to request build
    
    21
    +  job to be perform against the build farm.
    
    22
    +
    
    23
    +The `Remote Worker API`_ (RWAPI) describes another `gRPC`_ + `protocol-buffers`_
    
    24
    +interface that allows a central ``BotsService`` to manage a farm of pluggable workers.
    
    25
    +
    
    26
    +BuildGrid is combining these two interfaces in order to provide a complete
    
    27
    +remote caching and execution service. The high level architecture can be
    
    28
    +represented like this:
    
    29
    +
    
    30
    +.. graphviz::
    
    31
    +   :align: center
    
    32
    +
    
    33
    +    digraph remote_execution_overview {
    
    34
    +	node [shape = record,
    
    35
    +	      width=2,
    
    36
    +	      height=1];
    
    37
    +
    
    38
    +	ranksep = 2
    
    39
    +	compound=true
    
    40
    +	edge[arrowtail="vee"];
    
    41
    +	edge[arrowhead="vee"];
    
    42
    +
    
    43
    +	client [label = "Client",
    
    44
    +	color="#0342af",
    
    45
    +	fillcolor="#37c1e8",
    
    46
    +	style=filled,
    
    47
    +	shape=box]
    
    48
    +
    
    49
    +	subgraph cluster_controller{
    
    50
    +	    label = "Controller";
    
    51
    +	    labeljust = "c";
    
    52
    +	    fillcolor="#42edae";
    
    53
    +	    style=filled;
    
    54
    +	    controller [label = "{ExecutionService|BotsInterface\n}",
    
    55
    +			fillcolor="#17e86a",
    
    56
    +			style=filled];
    
    57
    +
    
    58
    +	}
    
    59
    +
    
    60
    +	subgraph cluster_worker0 {
    
    61
    +	    label = "Worker 1";
    
    62
    +	    labeljust = "c";
    
    63
    +	    color="#8e7747";
    
    64
    +	    fillcolor="#ffda8e";
    
    65
    +	    style=filled;
    
    66
    +	    bot0 [label = "{Bot|Host-tools}"
    
    67
    +		  fillcolor="#ffb214",
    
    68
    +		  style=filled];
    
    69
    +	}
    
    70
    +
    
    71
    +	subgraph cluster_worker1 {
    
    72
    +	    label = "Worker 2";
    
    73
    +	    labeljust = "c";
    
    74
    +	    color="#8e7747";
    
    75
    +	    fillcolor="#ffda8e";
    
    76
    +	    style=filled;
    
    77
    +	    bot1 [label = "{Bot|BuildBox}",
    
    78
    +		  fillcolor="#ffb214",
    
    79
    +		  style=filled];
    
    80
    +	}
    
    81
    +
    
    82
    +	client -> controller [
    
    83
    +	    dir = "both",
    
    84
    +	    headlabel = "REAPI",
    
    85
    +	    labelangle = 20.0,
    
    86
    +	    labeldistance = 9,
    
    87
    +	    labelfontsize = 15.0,
    
    88
    +	    lhead=cluster_controller];
    
    89
    +
    
    90
    +	controller -> bot0 [
    
    91
    +	    dir = "both",
    
    92
    +	    labelangle= 340.0,
    
    93
    +		labeldistance = 7.5,
    
    94
    +		labelfontsize = 15.0,
    
    95
    +	    taillabel = "RWAPI     ",
    
    96
    +	    lhead=cluster_worker0,
    
    97
    +	    ltail=cluster_controller];
    
    98
    +
    
    99
    +	controller -> bot1 [
    
    100
    +	    dir = "both",
    
    101
    +	    labelangle= 20.0,
    
    102
    +	    labeldistance = 7.5,
    
    103
    +	    labelfontsize = 15.0,
    
    104
    +		taillabel = "     RWAPI",
    
    105
    +	    lhead=cluster_worker1,
    
    106
    +	    ltail=cluster_controller];
    
    107
    +
    
    108
    +    }
    
    109
    +
    
    110
    +BuildGrid can be split up into separate endpoints. It is possible to have
    
    111
    +a separate ``ActionCache`` and ``CAS`` from the ``Controller``. The
    
    112
    +following diagram shows a typical setup.
    
    113
    +
    
    114
    +.. graphviz::
    
    115
    +   :align: center
    
    116
    +
    
    117
    +    digraph remote_execution_overview {
    
    118
    +
    
    119
    +	node [shape=record,
    
    120
    +	      width=2,
    
    121
    +	      height=1];
    
    122
    +
    
    123
    +	compound=true
    
    124
    +	graph [nodesep=1,
    
    125
    +	       ranksep=2]
    
    126
    +
    
    127
    +	edge[arrowtail="vee"];
    
    128
    +	edge[arrowhead="vee"];
    
    129
    +
    
    130
    +	client [label="Client",
    
    131
    +		color="#0342af",
    
    132
    +		fillcolor="#37c1e8",
    
    133
    +		style=filled,
    
    134
    +		shape=box]
    
    135
    +
    
    136
    +	cas [label="CAS",
    
    137
    +	     color="#840202",
    
    138
    +	     fillcolor="#c1034c",
    
    139
    +	     style=filled,
    
    140
    +	     shape=box]
    
    141
    +
    
    142
    +	subgraph cluster_controller{
    
    143
    +	    label="Controller";
    
    144
    +	    labeljust="c";
    
    145
    +	    fillcolor="#42edae";
    
    146
    +	    style=filled;
    
    147
    +	    controller [label="{ExecutionService|BotsInterface\n}",
    
    148
    +			fillcolor="#17e86a",
    
    149
    +			style=filled];
    
    150
    +
    
    151
    +	}
    
    152
    +
    
    153
    +	actioncache [label="ActionCache",
    
    154
    +		     color="#133f42",
    
    155
    +		     fillcolor="#219399",
    
    156
    +		     style=filled,
    
    157
    +		     shape=box]
    
    158
    +
    
    159
    +	subgraph cluster_worker0 {
    
    160
    +	    label="Worker";
    
    161
    +	    labeljust="c";
    
    162
    +	    color="#8e7747";
    
    163
    +	    fillcolor="#ffda8e";
    
    164
    +	    style=filled;
    
    165
    +	    bot0 [label="{Bot}"
    
    166
    +		  fillcolor="#ffb214",
    
    167
    +		  style=filled];
    
    168
    +	}
    
    169
    +
    
    170
    +	client -> controller [
    
    171
    +	    dir="both"];
    
    172
    +
    
    173
    +	client -> cas [
    
    174
    +	    dir="both",
    
    175
    +	    lhead=cluster_controller];
    
    176
    +
    
    177
    +	controller -> bot0 [
    
    178
    +	    dir="both",
    
    179
    +	    lhead=cluster_worker0];
    
    180
    +	    //ltail=cluster_controller];
    
    181
    +
    
    182
    +	cas -> bot0 [
    
    183
    +	    dir="both",
    
    184
    +	    lhead=cluster_worker0];
    
    185
    +
    
    186
    +	actioncache -> controller [
    
    187
    +	    dir="both"];
    
    188
    +
    
    189
    +	client -> actioncache [
    
    190
    +	    dir="both",
    
    191
    +	    constraint=false,
    
    192
    +    ];
    
    193
    +
    
    194
    +
    
    195
    +    }
    
    196
    +
    
    197
    +.. _Remote Execution API: https://github.com/bazelbuild/remote-apis/blob/master/build/bazel/remote/execution/v2
    
    198
    +.. _gRPC: https://grpc.io
    
    199
    +.. _protocol-buffers: https://developers.google.com/protocol-buffers
    
    200
    +.. _Remote Worker API: https://github.com/googleapis/googleapis/tree/master/google/devtools/remoteworkers/v1test2

  • docs/source/conf.py
    ... ... @@ -46,7 +46,8 @@ extensions = [
    46 46
         'sphinx.ext.autodoc',
    
    47 47
         'sphinx.ext.napoleon',
    
    48 48
         'sphinx_click.ext',
    
    49
    -    'sphinxcontrib.apidoc'
    
    49
    +    'sphinxcontrib.apidoc',
    
    50
    +    'sphinx.ext.graphviz',
    
    50 51
     ]
    
    51 52
     
    
    52 53
     # Add any paths that contain templates here, relative to this directory.
    

  • docs/source/index.rst
    ... ... @@ -19,6 +19,7 @@ Remote execution service implementing Google's REAPI and RWAPI.
    19 19
        using.rst
    
    20 20
        reference.rst
    
    21 21
        contributing.rst
    
    22
    +   architecture.rst
    
    22 23
        resources.rst
    
    23 24
     
    
    24 25
     
    



  • [Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]