[Notes] [Git][BuildGrid/buildgrid][santigl/100-server-capabilities] 14 c

Santiago Gil pushed to branch santigl/100-server-capabilities at BuildGrid / buildgrid

Commits:

c44ec8a6

by Martin Blanchard at 2019-02-13T11:44:11Z

Udapte .proto files to their latest version

Noteworthy changes include:
* From remote_execution.proto:
  - New message field in ExecuteResponse [1].
  - Output file and dir. parent folders clarifications [2].
  - Better doc. for do_not_cache/skip_cache_lookup [3].
  - Documention updates.
* From semver.proto:
  - Documentation updates.

[1] https://github.com/bazelbuild/remote-apis/pull/36
[2] https://github.com/bazelbuild/remote-apis/pull/42
[3] https://github.com/bazelbuild/remote-apis/pull/55

1c60ccc1

by Martin Blanchard at 2019-02-14T13:49:54Z

Make server instances aware of their name

https://gitlab.com/BuildGrid/buildgrid/issues/157

56f129ed

by Martin Blanchard at 2019-02-14T13:49:57Z

controller.py: Fix server instances registration

Delegate server registration to actual instance code instead of trying
to get it right from the controller class.

https://gitlab.com/BuildGrid/buildgrid/issues/157

f07c8895

by Martin Blanchard at 2019-02-14T13:50:03Z

scheduler.py: Hold parent's instance name

https://gitlab.com/BuildGrid/buildgrid/issues/157

4e96eb89

by Martin Blanchard at 2019-02-14T13:50:06Z

utils.py: Introduce a CAS browser URL helper

https://gitlab.com/BuildGrid/buildgrid/issues/157

3720db50

by Martin Blanchard at 2019-02-14T13:50:10Z

Introduce an action browser URL config. key

https://gitlab.com/BuildGrid/buildgrid/issues/157

f768c632

by Martin Blanchard at 2019-02-14T13:50:14Z

Generate per-job action browser URLs

https://gitlab.com/BuildGrid/buildgrid/issues/157

d0a8c0bc

by Martin Blanchard at 2019-02-14T13:50:17Z

tests: Add simple tests utils and helpers

https://gitlab.com/BuildGrid/buildgrid/issues/157

78efe8c6

by Santiago Gil at 2019-02-18T16:32:52Z

Scheduler: add mutex to job queue

This solves race conditions in the scheduler (introduced by !151, which
seeked to address #104).

ed7ab756

by Martin Blanchard at 2019-02-19T16:23:03Z

capabilities/service.py: Log debug message on request

https://gitlab.com/BuildGrid/buildgrid/issues/161

95f83758

by Martin Blanchard at 2019-02-19T16:23:19Z

settings.py: Add supported versions for REAPI

https://gitlab.com/BuildGrid/buildgrid/issues/161

b9f59bf3

by Martin Blanchard at 2019-02-19T16:25:13Z

capabilities/instance.py: Advertise supported REAPI version

https://gitlab.com/BuildGrid/buildgrid/issues/161

4c971b76

by Santiago Gil at 2019-02-20T10:46:59Z

Scheduler: fix monitoring variables initialization

Calling `Scheduler.activate_monitoring()` from the `Scheduler`
constructor didn't have any effect because `self.activate_monitoring()`
always exited early.

This change fixes the boolean check before that function call so that
the monitoring structures get initialized properly.

4796f913

by Santiago Gil at 2019-02-20T15:24:09Z

CAS.{Up, Down}loader(): if provided, respect `CacheCapabilities`

Try fetching the `CacheCapabilities.max_batch_total_size_bytes` value
from the server and, if the server provides a limit, respect it for
batch transactions.

Also, make the threshold value for determining when to queue a file
for fetching in a batch request a configurable % of the maximum
batch request size.

(Addresses #100)

23 changed files:

buildgrid/_app/settings/parser.py
buildgrid/_app/settings/reference.yml
buildgrid/_protos/build/bazel/remote/execution/v2/remote_execution.proto
buildgrid/_protos/build/bazel/remote/execution/v2/remote_execution_pb2.py
buildgrid/_protos/build/bazel/remote/execution/v2/remote_execution_pb2_grpc.py
buildgrid/_protos/build/bazel/semver/semver.proto
buildgrid/client/cas.py
buildgrid/server/actioncache/storage.py
buildgrid/server/bots/instance.py
buildgrid/server/capabilities/instance.py
buildgrid/server/capabilities/service.py
buildgrid/server/cas/instance.py
buildgrid/server/controller.py
buildgrid/server/execution/instance.py
buildgrid/server/job.py
buildgrid/server/operations/instance.py
buildgrid/server/referencestorage/storage.py
buildgrid/server/scheduler.py
buildgrid/settings.py
buildgrid/utils.py
tests/integration/execution_service.py
tests/integration/operations_service.py
+ tests/test_utils.py

Changes:

buildgrid/_app/settings/parser.py

@@ -235,8 +235,8 @@ class Execution(YamlFactory):
      yaml_tag = u'!execution'
 -    def __new__(cls, storage, action_cache=None):
 -        return ExecutionController(action_cache, storage)
 +    def __new__(cls, storage, action_cache=None, action_browser_url=None):
 +        return ExecutionController(storage, action_cache, action_browser_url)
  class Action(YamlFactory):

buildgrid/_app/settings/reference.yml

@@ -75,7 +75,7 @@ instances:
          # Whether or not writing to the cache is allowed.
          allow-updates: true
          ##
 -        # Whether failed actions (non-zero exit code) are stored
 +        # Whether failed actions (non-zero exit code) are stored.
          cache-failed-actions: true
        - !execution
@@ -85,6 +85,9 @@ instances:
          ##
          # Alias to an action-cache service.
          action-cache: *main-action
 +        ##
 +        # Base URL for external build action (web) browser service.
 +        action-browser-url: http://localhost:8080
        - !cas
          ##

buildgrid/_protos/build/bazel/remote/execution/v2/remote_execution.proto

@@ -81,6 +81,7 @@ service Execution {
    // action will be reported in the `status` field of the `ExecuteResponse`. The
    // server MUST NOT set the `error` field of the `Operation` proto.
    // The possible errors include:
 +  //
    // * `INVALID_ARGUMENT`: One or more arguments are invalid.
    // * `FAILED_PRECONDITION`: One or more errors occurred in setting up the
    //   action requested, such as a missing input or command or no worker being
@@ -140,6 +141,7 @@ service ActionCache {
    // Retrieve a cached execution result.
    //
    // Errors:
 +  //
    // * `NOT_FOUND`: The requested `ActionResult` is not in the cache.
    rpc GetActionResult(GetActionResultRequest) returns (ActionResult) {
      option (google.api.http) = { get: "/v2/{instance_name=**}/actionResults/{action_digest.hash}/{action_digest.size_bytes}" };
@@ -147,11 +149,6 @@ service ActionCache {
    // Upload a new execution result.
    //
 -  // This method is intended for servers which implement the distributed cache
 -  // independently of the
 -  // [Execution][build.bazel.remote.execution.v2.Execution] API. As a
 -  // result, it is OPTIONAL for servers to implement.
 -  //
    // In order to allow the server to perform access control based on the type of
    // action, and to assist with client debugging, the client MUST first upload
    // the [Action][build.bazel.remote.execution.v2.Execution] that produced the
@@ -160,7 +157,10 @@ service ActionCache {
    // `ContentAddressableStorage`.
    //
    // Errors:
 -  // * `UNIMPLEMENTED`: This method is not supported by the server.
 +  //
 +  // * `INVALID_ARGUMENT`: One or more arguments are invalid.
 +  // * `FAILED_PRECONDITION`: One or more errors occurred in updating the
 +  //   action result, such as a missing command or action.
    // * `RESOURCE_EXHAUSTED`: There is insufficient storage space to add the
    //   entry to the cache.
    rpc UpdateActionResult(UpdateActionResultRequest) returns (ActionResult) {
@@ -207,6 +207,9 @@ service ActionCache {
  // by the server. For servers which do not support multiple instances, then the
  // `instance_name` is the empty path and the leading slash is omitted, so that
  // the `resource_name` becomes `uploads/{uuid}/blobs/{hash}/{size}`.
 +// To simplify parsing, a path segment cannot equal any of the following
 +// keywords: `blobs`, `uploads`, `actions`, `actionResults`, `operations` and
 +// `capabilities`.
  //
  // When attempting an upload, if another client has already completed the upload
  // (which may occur in the middle of a single upload if another client uploads
@@ -258,10 +261,12 @@ service ContentAddressableStorage {
    // independently.
    //
    // Errors:
 +  //
    // * `INVALID_ARGUMENT`: The client attempted to upload more than the
    //   server supported limit.
    //
    // Individual requests may return the following errors, additionally:
 +  //
    // * `RESOURCE_EXHAUSTED`: There is insufficient disk quota to store the blob.
    // * `INVALID_ARGUMENT`: The
    // [Digest][build.bazel.remote.execution.v2.Digest] does not match the
@@ -284,6 +289,7 @@ service ContentAddressableStorage {
    // independently.
    //
    // Errors:
 +  //
    // * `INVALID_ARGUMENT`: The client attempted to read more than the
    //   server supported limit.
    //
@@ -390,7 +396,8 @@ message Action {
    // immediately, rather than whenever the cache entry gets evicted.
    google.protobuf.Duration timeout = 6;
 -  // If true, then the `Action`'s result cannot be cached.
 +  // If true, then the `Action`'s result cannot be cached, and in-flight
 +  // requests for the same `Action` may not be merged.
    bool do_not_cache = 7;
+ }
@@ -421,7 +428,8 @@ message Command {
    // provide its own default environment variables; these defaults can be
    // overridden using this field. Additional variables can also be specified.
    //
 -  // In order to ensure that equivalent `Command`s always hash to the same
 +  // In order to ensure that equivalent
 +  // [Command][build.bazel.remote.execution.v2.Command]s always hash to the same
    // value, the environment variables MUST be lexicographically sorted by name.
    // Sorting of strings is done by code point, equivalently, by the UTF-8 bytes.
    repeated EnvironmentVariable environment_variables = 2;
@@ -444,6 +452,9 @@ message Command {
    //
    // An output file cannot be duplicated, be a parent of another output file, or
    // have the same path as any of the listed output directories.
 +  //
 +  // Directories leading up to the output files are created by the worker prior
 +  // to execution, even if they are not explicitly part of the input root.
    repeated string output_files = 3;
    // A list of the output directories that the client expects to retrieve from
@@ -468,6 +479,10 @@ message Command {
    //
    // An output directory cannot be duplicated or have the same path as any of
    // the listed output files.
 +  //
 +  // Directories leading up to the output directories (but not the output
 +  // directories themselves) are created by the worker prior to execution, even
 +  // if they are not explicitly part of the input root.
    repeated string output_directories = 4;
    // The platform requirements for the execution environment. The server MAY
@@ -531,12 +546,18 @@ message Platform {
  // In order to ensure that two equivalent directory trees hash to the same
  // value, the following restrictions MUST be obeyed when constructing a
  // a `Directory`:
 -//   - Every child in the directory must have a path of exactly one segment.
 -//     Multiple levels of directory hierarchy may not be collapsed.
 -//   - Each child in the directory must have a unique path segment (file name).
 -//   - The files, directories and symlinks in the directory must each be sorted
 -//     in lexicographical order by path. The path strings must be sorted by code
 -//     point, equivalently, by UTF-8 bytes.
 +//
 +// * Every child in the directory must have a path of exactly one segment.
 +//   Multiple levels of directory hierarchy may not be collapsed.
 +// * Each child in the directory must have a unique path segment (file name).
 +//   Note that while the API itself is case-sensitive, the environment where
 +//   the Action is executed may or may not be case-sensitive. That is, it is
 +//   legal to call the API with a Directory that has both "Foo" and "foo" as
 +//   children, but the Action may be rejected by the remote system upon
 +//   execution.
 +// * The files, directories and symlinks in the directory must each be sorted
 +//   in lexicographical order by path. The path strings must be sorted by code
 +//   point, equivalently, by UTF-8 bytes.
  //
  // A `Directory` that obeys the restrictions is said to be in canonical form.
  //
@@ -656,11 +677,12 @@ message SymlinkNode {
  // When a `Digest` is used to refer to a proto message, it always refers to the
  // message in binary encoded form. To ensure consistent hashing, clients and
  // servers MUST ensure that they serialize messages according to the following
 -// rules, even if there are alternate valid encodings for the same message.
 -// - Fields are serialized in tag order.
 -// - There are no unknown fields.
 -// - There are no duplicate fields.
 -// - Fields are serialized according to the default semantics for their type.
 +// rules, even if there are alternate valid encodings for the same message:
 +//
 +// * Fields are serialized in tag order.
 +// * There are no unknown fields.
 +// * There are no duplicate fields.
 +// * Fields are serialized according to the default semantics for their type.
  //
  // Most protocol buffer implementations will always follow these rules when
  // serializing, but care should be taken to avoid shortcuts. For instance,
@@ -727,7 +749,7 @@ message ActionResult {
    // The output files of the action that are symbolic links to other files. Those
    // may be links to other output files, or input files, or even absolute paths
    // outside of the working directory, if the server supports
 -  // [SymlinkAbsolutePathStrategy.ALLOWED][build.bazel.remote.execution.v2.SymlinkAbsolutePathStrategy].
 +  // [SymlinkAbsolutePathStrategy.ALLOWED][build.bazel.remote.execution.v2.CacheCapabilities.SymlinkAbsolutePathStrategy].
    // For each output file requested in the `output_files` field of the Action,
    // if the corresponding file existed after
    // the action completed, a single entry will be present either in this field,
@@ -804,7 +826,7 @@ message ActionResult {
    // directories. Those may be links to other output directories, or input
    // directories, or even absolute paths outside of the working directory,
    // if the server supports
 -  // [SymlinkAbsolutePathStrategy.ALLOWED][build.bazel.remote.execution.v2.SymlinkAbsolutePathStrategy].
 +  // [SymlinkAbsolutePathStrategy.ALLOWED][build.bazel.remote.execution.v2.CacheCapabilities.SymlinkAbsolutePathStrategy].
    // For each output directory requested in the `output_directories` field of
    // the Action, if the directory file existed after
    // the action completed, a single entry will be present either in this field,
@@ -961,9 +983,19 @@ message ExecuteRequest {
    // omitted.
    string instance_name = 1;
 -  // If true, the action will be executed anew even if its result was already
 -  // present in the cache. If false, the result may be served from the
 -  // [ActionCache][build.bazel.remote.execution.v2.ActionCache].
 +  // If true, the action will be executed even if its result is already
 +  // present in the [ActionCache][build.bazel.remote.execution.v2.ActionCache].
 +  // The execution is still allowed to be merged with other in-flight executions
 +  // of the same action, however - semantically, the service MUST only guarantee
 +  // that the results of an execution with this field set were not visible
 +  // before the corresponding execution request was sent.
 +  // Note that actions from execution requests setting this field set are still
 +  // eligible to be entered into the action cache upon completion, and services
 +  // SHOULD overwrite any existing entries that may exist. This allows
 +  // skip_cache_lookup requests to be used as a mechanism for replacing action
 +  // cache entries that reference outputs no longer available or that are
 +  // poisoned in any way.
 +  // If false, the result may be served from the action cache.
    bool skip_cache_lookup = 3;
    reserved 2, 4, 5; // Used for removed fields in an earlier version of the API.
@@ -1027,6 +1059,10 @@ message ExecuteResponse {
    // phase. The keys SHOULD be human readable so that a client can display them
    // to a user.
    map<string, LogFile> server_logs = 4;
++
 +  // Freeform informational message with details on the execution of the action
 +  // that may be displayed to the user upon failure or when requested explicitly.
 +  string message = 5;
+ }
  // Metadata about an ongoing
@@ -1072,7 +1108,7 @@ message ExecuteOperationMetadata {
  // A request message for
  // [WaitExecution][build.bazel.remote.execution.v2.Execution.WaitExecution].
  message WaitExecutionRequest {
 -  // The name of the [Operation][google.longrunning.operations.v1.Operation]
 +  // The name of the [Operation][google.longrunning.Operation]
    // returned by [Execute][build.bazel.remote.execution.v2.Execution.Execute].
    string name = 1;
+ }
@@ -1193,7 +1229,7 @@ message BatchReadBlobsRequest {
  // A response message for
  // [ContentAddressableStorage.BatchReadBlobs][build.bazel.remote.execution.v2.ContentAddressableStorage.BatchReadBlobs].
  message BatchReadBlobsResponse {
 -  // A response corresponding to a single blob that the client tried to upload.
 +  // A response corresponding to a single blob that the client tried to download.
    message Response {
      // The digest to which this response corresponds.
      Digest digest = 1;
@@ -1251,7 +1287,7 @@ message GetTreeResponse {
+ }
  // A request message for
 -// [Capabilities.GetCapabilities][google.devtools.remoteexecution.v2.Capabilities.GetCapabilities].
 +// [Capabilities.GetCapabilities][build.bazel.remote.execution.v2.Capabilities.GetCapabilities].
  message GetCapabilitiesRequest {
    // The instance of the execution system to operate against. A server may
    // support multiple instances of the execution system (with their own workers,
@@ -1262,7 +1298,7 @@ message GetCapabilitiesRequest {
+ }
  // A response message for
 -// [Capabilities.GetCapabilities][google.devtools.remoteexecution.v2.Capabilities.GetCapabilities].
 +// [Capabilities.GetCapabilities][build.bazel.remote.execution.v2.Capabilities.GetCapabilities].
  message ServerCapabilities {
    // Capabilities of the remote cache system.
    CacheCapabilities cache_capabilities = 1;
@@ -1283,9 +1319,16 @@ message ServerCapabilities {
  // The digest function used for converting values into keys for CAS and Action
  // Cache.
  enum DigestFunction {
 +  // It is an error for the server to return this value.
    UNKNOWN = 0;
++
 +  // The Sha-256 digest function.
    SHA256 = 1;
++
 +  // The Sha-1 digest function.
    SHA1 = 2;
++
 +  // The MD5 digest function.
    MD5 = 3;
+ }
@@ -1312,9 +1355,10 @@ message CacheCapabilities {
    enum SymlinkAbsolutePathStrategy {
      UNKNOWN = 0;
 -    // Server will return an INVALID_ARGUMENT on input symlinks with absolute targets.
 +    // Server will return an `INVALID_ARGUMENT` on input symlinks with absolute
 +    // targets.
      // If an action tries to create an output symlink with an absolute target, a
 -    // FAILED_PRECONDITION will be returned.
 +    // `FAILED_PRECONDITION` will be returned.
      DISALLOWED = 1;
      // Server will allow symlink targets to escape the input root tree, possibly
@@ -1367,8 +1411,9 @@ message ToolDetails {
  // external context of the request. The server may use this for logging or other
  // purposes. To use it, the client attaches the header to the call using the
  // canonical proto serialization:
 -// name: build.bazel.remote.execution.v2.requestmetadata-bin
 -// contents: the base64 encoded binary RequestMetadata message.
 +//
 +// * name: `build.bazel.remote.execution.v2.requestmetadata-bin`
 +// * contents: the base64 encoded binary `RequestMetadata` message.
  message RequestMetadata {
    // The details for the tool invoking the requests.
    ToolDetails tool_details = 1;

buildgrid/_protos/build/bazel/remote/execution/v2/remote_execution_pb2.py

@@ -26,7 +26,7 @@ DESCRIPTOR = _descriptor.FileDescriptor(
    package='build.bazel.remote.execution.v2',
    syntax='proto3',
    serialized_options=_b('\n\037build.bazel.remote.execution.v2B\024RemoteExecutionProtoP\001Z\017remoteexecution\242\002\003REX\252\002\037Build.Bazel.Remote.Execution.V2'),
 -  serialized_pb=_b('\n6build/bazel/remote/execution/v2/remote_execution.proto\x12\x1f\x62uild.bazel.remote.execution.v2\x1a\x1f\x62uild/bazel/semver/semver.proto\x1a\x1cgoogle/api/annotations.proto\x1a#google/longrunning/operations.proto\x1a\x1egoogle/protobuf/duration.proto\x1a\x1fgoogle/protobuf/timestamp.proto\x1a\x17google/rpc/status.proto\"\xd5\x01\n\x06\x41\x63tion\x12?\n\x0e\x63ommand_digest\x18\x01 \x01(\x0b\x32\'.build.bazel.remote.execution.v2.Digest\x12\x42\n\x11input_root_digest\x18\x02 \x01(\x0b\x32\'.build.bazel.remote.execution.v2.Digest\x12*\n\x07timeout\x18\x06 \x01(\x0b\x32\x19.google.protobuf.Duration\x12\x14\n\x0c\x64o_not_cache\x18\x07 \x01(\x08J\x04\x08\x03\x10\x06\"\xb7\x02\n\x07\x43ommand\x12\x11\n\targuments\x18\x01 \x03(\t\x12[\n\x15\x65nvironment_variables\x18\x02 \x03(\x0b\x32<.build.bazel.remote.execution.v2.Command.EnvironmentVariable\x12\x14\n\x0coutput_files\x18\x03 \x03(\t\x12\x1a\n\x12output_directories\x18\x04 \x03(\t\x12;\n\x08platform\x18\x05 \x01(\x0b\x32).build.bazel.remote.execution.v2.Platform\x12\x19\n\x11working_directory\x18\x06 \x01(\t\x1a\x32\n\x13\x45nvironmentVariable\x12\x0c\n\x04name\x18\x01 \x01(\t\x12\r\n\x05value\x18\x02 \x01(\t\"{\n\x08Platform\x12\x46\n\nproperties\x18\x01 \x03(\x0b\x32\x32.build.bazel.remote.execution.v2.Platform.Property\x1a\'\n\x08Property\x12\x0c\n\x04name\x18\x01 \x01(\t\x12\r\n\x05value\x18\x02 \x01(\t\"\xca\x01\n\tDirectory\x12\x38\n\x05\x66iles\x18\x01 \x03(\x0b\x32).build.bazel.remote.execution.v2.FileNode\x12\x43\n\x0b\x64irectories\x18\x02 \x03(\x0b\x32..build.bazel.remote.execution.v2.DirectoryNode\x12>\n\x08symlinks\x18\x03 \x03(\x0b\x32,.build.bazel.remote.execution.v2.SymlinkNode\"n\n\x08\x46ileNode\x12\x0c\n\x04name\x18\x01 \x01(\t\x12\x37\n\x06\x64igest\x18\x02 \x01(\x0b\x32\'.build.bazel.remote.execution.v2.Digest\x12\x15\n\ris_executable\x18\x04 \x01(\x08J\x04\x08\x03\x10\x04\"V\n\rDirectoryNode\x12\x0c\n\x04name\x18\x01 \x01(\t\x12\x37\n\x06\x64igest\x18\x02 \x01(\x0b\x32\'.build.bazel.remote.execution.v2.Digest\"+\n\x0bSymlinkNode\x12\x0c\n\x04name\x18\x01 \x01(\t\x12\x0e\n\x06target\x18\x02 \x01(\t\"*\n\x06\x44igest\x12\x0c\n\x04hash\x18\x01 \x01(\t\x12\x12\n\nsize_bytes\x18\x02 \x01(\x03\"\xec\x04\n\x16\x45xecutedActionMetadata\x12\x0e\n\x06worker\x18\x01 \x01(\t\x12\x34\n\x10queued_timestamp\x18\x02 \x01(\x0b\x32\x1a.google.protobuf.Timestamp\x12:\n\x16worker_start_timestamp\x18\x03 \x01(\x0b\x32\x1a.google.protobuf.Timestamp\x12>\n\x1aworker_completed_timestamp\x18\x04 \x01(\x0b\x32\x1a.google.protobuf.Timestamp\x12?\n\x1binput_fetch_start_timestamp\x18\x05 \x01(\x0b\x32\x1a.google.protobuf.Timestamp\x12\x43\n\x1finput_fetch_completed_timestamp\x18\x06 \x01(\x0b\x32\x1a.google.protobuf.Timestamp\x12=\n\x19\x65xecution_start_timestamp\x18\x07 \x01(\x0b\x32\x1a.google.protobuf.Timestamp\x12\x41\n\x1d\x65xecution_completed_timestamp\x18\x08 \x01(\x0b\x32\x1a.google.protobuf.Timestamp\x12\x41\n\x1doutput_upload_start_timestamp\x18\t \x01(\x0b\x32\x1a.google.protobuf.Timestamp\x12\x45\n!output_upload_completed_timestamp\x18\n \x01(\x0b\x32\x1a.google.protobuf.Timestamp\"\xd6\x04\n\x0c\x41\x63tionResult\x12\x41\n\x0coutput_files\x18\x02 \x03(\x0b\x32+.build.bazel.remote.execution.v2.OutputFile\x12L\n\x14output_file_symlinks\x18\n \x03(\x0b\x32..build.bazel.remote.execution.v2.OutputSymlink\x12L\n\x12output_directories\x18\x03 \x03(\x0b\x32\x30.build.bazel.remote.execution.v2.OutputDirectory\x12Q\n\x19output_directory_symlinks\x18\x0b \x03(\x0b\x32..build.bazel.remote.execution.v2.OutputSymlink\x12\x11\n\texit_code\x18\x04 \x01(\x05\x12\x12\n\nstdout_raw\x18\x05 \x01(\x0c\x12>\n\rstdout_digest\x18\x06 \x01(\x0b\x32\'.build.bazel.remote.execution.v2.Digest\x12\x12\n\nstderr_raw\x18\x07 \x01(\x0c\x12>\n\rstderr_digest\x18\x08 \x01(\x0b\x32\'.build.bazel.remote.execution.v2.Digest\x12S\n\x12\x65xecution_metadata\x18\t \x01(\x0b\x32\x37.build.bazel.remote.execution.v2.ExecutedActionMetadataJ\x04\x08\x01\x10\x02\"p\n\nOutputFile\x12\x0c\n\x04path\x18\x01 \x01(\t\x12\x37\n\x06\x64igest\x18\x02 \x01(\x0b\x32\'.build.bazel.remote.execution.v2.Digest\x12\x15\n\ris_executable\x18\x04 \x01(\x08J\x04\x08\x03\x10\x04\"~\n\x04Tree\x12\x38\n\x04root\x18\x01 \x01(\x0b\x32*.build.bazel.remote.execution.v2.Directory\x12<\n\x08\x63hildren\x18\x02 \x03(\x0b\x32*.build.bazel.remote.execution.v2.Directory\"c\n\x0fOutputDirectory\x12\x0c\n\x04path\x18\x01 \x01(\t\x12<\n\x0btree_digest\x18\x03 \x01(\x0b\x32\'.build.bazel.remote.execution.v2.DigestJ\x04\x08\x02\x10\x03\"-\n\rOutputSymlink\x12\x0c\n\x04path\x18\x01 \x01(\t\x12\x0e\n\x06target\x18\x02 \x01(\t\"#\n\x0f\x45xecutionPolicy\x12\x10\n\x08priority\x18\x01 \x01(\x05\"&\n\x12ResultsCachePolicy\x12\x10\n\x08priority\x18\x01 \x01(\x05\"\xb3\x02\n\x0e\x45xecuteRequest\x12\x15\n\rinstance_name\x18\x01 \x01(\t\x12\x19\n\x11skip_cache_lookup\x18\x03 \x01(\x08\x12>\n\raction_digest\x18\x06 \x01(\x0b\x32\'.build.bazel.remote.execution.v2.Digest\x12J\n\x10\x65xecution_policy\x18\x07 \x01(\x0b\x32\x30.build.bazel.remote.execution.v2.ExecutionPolicy\x12Q\n\x14results_cache_policy\x18\x08 \x01(\x0b\x32\x33.build.bazel.remote.execution.v2.ResultsCachePolicyJ\x04\x08\x02\x10\x03J\x04\x08\x04\x10\x05J\x04\x08\x05\x10\x06\"Z\n\x07LogFile\x12\x37\n\x06\x64igest\x18\x01 \x01(\x0b\x32\'.build.bazel.remote.execution.v2.Digest\x12\x16\n\x0ehuman_readable\x18\x02 \x01(\x08\"\xbf\x02\n\x0f\x45xecuteResponse\x12=\n\x06result\x18\x01 \x01(\x0b\x32-.build.bazel.remote.execution.v2.ActionResult\x12\x15\n\rcached_result\x18\x02 \x01(\x08\x12\"\n\x06status\x18\x03 \x01(\x0b\x32\x12.google.rpc.Status\x12U\n\x0bserver_logs\x18\x04 \x03(\x0b\x32@.build.bazel.remote.execution.v2.ExecuteResponse.ServerLogsEntry\x1a[\n\x0fServerLogsEntry\x12\x0b\n\x03key\x18\x01 \x01(\t\x12\x37\n\x05value\x18\x02 \x01(\x0b\x32(.build.bazel.remote.execution.v2.LogFile:\x02\x38\x01\"\xb3\x02\n\x18\x45xecuteOperationMetadata\x12N\n\x05stage\x18\x01 \x01(\x0e\x32?.build.bazel.remote.execution.v2.ExecuteOperationMetadata.Stage\x12>\n\raction_digest\x18\x02 \x01(\x0b\x32\'.build.bazel.remote.execution.v2.Digest\x12\x1a\n\x12stdout_stream_name\x18\x03 \x01(\t\x12\x1a\n\x12stderr_stream_name\x18\x04 \x01(\t\"O\n\x05Stage\x12\x0b\n\x07UNKNOWN\x10\x00\x12\x0f\n\x0b\x43\x41\x43HE_CHECK\x10\x01\x12\n\n\x06QUEUED\x10\x02\x12\r\n\tEXECUTING\x10\x03\x12\r\n\tCOMPLETED\x10\x04\"$\n\x14WaitExecutionRequest\x12\x0c\n\x04name\x18\x01 \x01(\t\"o\n\x16GetActionResultRequest\x12\x15\n\rinstance_name\x18\x01 \x01(\t\x12>\n\raction_digest\x18\x02 \x01(\x0b\x32\'.build.bazel.remote.execution.v2.Digest\"\x8b\x02\n\x19UpdateActionResultRequest\x12\x15\n\rinstance_name\x18\x01 \x01(\t\x12>\n\raction_digest\x18\x02 \x01(\x0b\x32\'.build.bazel.remote.execution.v2.Digest\x12\x44\n\raction_result\x18\x03 \x01(\x0b\x32-.build.bazel.remote.execution.v2.ActionResult\x12Q\n\x14results_cache_policy\x18\x04 \x01(\x0b\x32\x33.build.bazel.remote.execution.v2.ResultsCachePolicy\"o\n\x17\x46indMissingBlobsRequest\x12\x15\n\rinstance_name\x18\x01 \x01(\t\x12=\n\x0c\x62lob_digests\x18\x02 \x03(\x0b\x32\'.build.bazel.remote.execution.v2.Digest\"a\n\x18\x46indMissingBlobsResponse\x12\x45\n\x14missing_blob_digests\x18\x02 \x03(\x0b\x32\'.build.bazel.remote.execution.v2.Digest\"\xd6\x01\n\x17\x42\x61tchUpdateBlobsRequest\x12\x15\n\rinstance_name\x18\x01 \x01(\t\x12R\n\x08requests\x18\x02 \x03(\x0b\x32@.build.bazel.remote.execution.v2.BatchUpdateBlobsRequest.Request\x1aP\n\x07Request\x12\x37\n\x06\x64igest\x18\x01 \x01(\x0b\x32\'.build.bazel.remote.execution.v2.Digest\x12\x0c\n\x04\x64\x61ta\x18\x02 \x01(\x0c\"\xda\x01\n\x18\x42\x61tchUpdateBlobsResponse\x12U\n\tresponses\x18\x01 \x03(\x0b\x32\x42.build.bazel.remote.execution.v2.BatchUpdateBlobsResponse.Response\x1ag\n\x08Response\x12\x37\n\x06\x64igest\x18\x01 \x01(\x0b\x32\'.build.bazel.remote.execution.v2.Digest\x12\"\n\x06status\x18\x02 \x01(\x0b\x32\x12.google.rpc.Status\"h\n\x15\x42\x61tchReadBlobsRequest\x12\x15\n\rinstance_name\x18\x01 \x01(\t\x12\x38\n\x07\x64igests\x18\x02 \x03(\x0b\x32\'.build.bazel.remote.execution.v2.Digest\"\xe4\x01\n\x16\x42\x61tchReadBlobsResponse\x12S\n\tresponses\x18\x01 \x03(\x0b\x32@.build.bazel.remote.execution.v2.BatchReadBlobsResponse.Response\x1au\n\x08Response\x12\x37\n\x06\x64igest\x18\x01 \x01(\x0b\x32\'.build.bazel.remote.execution.v2.Digest\x12\x0c\n\x04\x64\x61ta\x18\x02 \x01(\x0c\x12\"\n\x06status\x18\x03 \x01(\x0b\x32\x12.google.rpc.Status\"\x8c\x01\n\x0eGetTreeRequest\x12\x15\n\rinstance_name\x18\x01 \x01(\t\x12<\n\x0broot_digest\x18\x02 \x01(\x0b\x32\'.build.bazel.remote.execution.v2.Digest\x12\x11\n\tpage_size\x18\x03 \x01(\x05\x12\x12\n\npage_token\x18\x04 \x01(\t\"k\n\x0fGetTreeResponse\x12?\n\x0b\x64irectories\x18\x01 \x03(\x0b\x32*.build.bazel.remote.execution.v2.Directory\x12\x17\n\x0fnext_page_token\x18\x02 \x01(\t\"/\n\x16GetCapabilitiesRequest\x12\x15\n\rinstance_name\x18\x01 \x01(\t\"\xe3\x02\n\x12ServerCapabilities\x12N\n\x12\x63\x61\x63he_capabilities\x18\x01 \x01(\x0b\x32\x32.build.bazel.remote.execution.v2.CacheCapabilities\x12V\n\x16\x65xecution_capabilities\x18\x02 \x01(\x0b\x32\x36.build.bazel.remote.execution.v2.ExecutionCapabilities\x12:\n\x16\x64\x65precated_api_version\x18\x03 \x01(\x0b\x32\x1a.build.bazel.semver.SemVer\x12\x33\n\x0flow_api_version\x18\x04 \x01(\x0b\x32\x1a.build.bazel.semver.SemVer\x12\x34\n\x10high_api_version\x18\x05 \x01(\x0b\x32\x1a.build.bazel.semver.SemVer\"7\n\x1d\x41\x63tionCacheUpdateCapabilities\x12\x16\n\x0eupdate_enabled\x18\x01 \x01(\x08\"\xac\x01\n\x14PriorityCapabilities\x12W\n\npriorities\x18\x01 \x03(\x0b\x32\x43.build.bazel.remote.execution.v2.PriorityCapabilities.PriorityRange\x1a;\n\rPriorityRange\x12\x14\n\x0cmin_priority\x18\x01 \x01(\x05\x12\x14\n\x0cmax_priority\x18\x02 \x01(\x05\"\x88\x04\n\x11\x43\x61\x63heCapabilities\x12H\n\x0f\x64igest_function\x18\x01 \x03(\x0e\x32/.build.bazel.remote.execution.v2.DigestFunction\x12h\n action_cache_update_capabilities\x18\x02 \x01(\x0b\x32>.build.bazel.remote.execution.v2.ActionCacheUpdateCapabilities\x12Z\n\x1b\x63\x61\x63he_priority_capabilities\x18\x03 \x01(\x0b\x32\x35.build.bazel.remote.execution.v2.PriorityCapabilities\x12\"\n\x1amax_batch_total_size_bytes\x18\x04 \x01(\x03\x12v\n\x1esymlink_absolute_path_strategy\x18\x05 \x01(\x0e\x32N.build.bazel.remote.execution.v2.CacheCapabilities.SymlinkAbsolutePathStrategy\"G\n\x1bSymlinkAbsolutePathStrategy\x12\x0b\n\x07UNKNOWN\x10\x00\x12\x0e\n\nDISALLOWED\x10\x01\x12\x0b\n\x07\x41LLOWED\x10\x02\"\xd7\x01\n\x15\x45xecutionCapabilities\x12H\n\x0f\x64igest_function\x18\x01 \x01(\x0e\x32/.build.bazel.remote.execution.v2.DigestFunction\x12\x14\n\x0c\x65xec_enabled\x18\x02 \x01(\x08\x12^\n\x1f\x65xecution_priority_capabilities\x18\x03 \x01(\x0b\x32\x35.build.bazel.remote.execution.v2.PriorityCapabilities\"6\n\x0bToolDetails\x12\x11\n\ttool_name\x18\x01 \x01(\t\x12\x14\n\x0ctool_version\x18\x02 \x01(\t\"\xa7\x01\n\x0fRequestMetadata\x12\x42\n\x0ctool_details\x18\x01 \x01(\x0b\x32,.build.bazel.remote.execution.v2.ToolDetails\x12\x11\n\taction_id\x18\x02 \x01(\t\x12\x1a\n\x12tool_invocation_id\x18\x03 \x01(\t\x12!\n\x19\x63orrelated_invocations_id\x18\x04 \x01(\t*<\n\x0e\x44igestFunction\x12\x0b\n\x07UNKNOWN\x10\x00\x12\n\n\x06SHA256\x10\x01\x12\x08\n\x04SHA1\x10\x02\x12\x07\n\x03MD5\x10\x03\x32\xb9\x02\n\tExecution\x12\x8e\x01\n\x07\x45xecute\x12/.build.bazel.remote.execution.v2.ExecuteRequest\x1a\x1d.google.longrunning.Operation\"1\x82\xd3\xe4\x93\x02+\"&/v2/{instance_name=**}/actions:execute:\x01*0\x01\x12\x9a\x01\n\rWaitExecution\x12\x35.build.bazel.remote.execution.v2.WaitExecutionRequest\x1a\x1d.google.longrunning.Operation\"1\x82\xd3\xe4\x93\x02+\"&/v2/{name=operations/**}:waitExecution:\x01*0\x01\x32\xd6\x03\n\x0b\x41\x63tionCache\x12\xd7\x01\n\x0fGetActionResult\x12\x37.build.bazel.remote.execution.v2.GetActionResultRequest\x1a-.build.bazel.remote.execution.v2.ActionResult\"\\\x82\xd3\xe4\x93\x02V\x12T/v2/{instance_name=**}/actionResults/{action_digest.hash}/{action_digest.size_bytes}\x12\xec\x01\n\x12UpdateActionResult\x12:.build.bazel.remote.execution.v2.UpdateActionResultRequest\x1a-.build.bazel.remote.execution.v2.ActionResult\"k\x82\xd3\xe4\x93\x02\x65\x1aT/v2/{instance_name=**}/actionResults/{action_digest.hash}/{action_digest.size_bytes}:\raction_result2\x9b\x06\n\x19\x43ontentAddressableStorage\x12\xbc\x01\n\x10\x46indMissingBlobs\x12\x38.build.bazel.remote.execution.v2.FindMissingBlobsRequest\x1a\x39.build.bazel.remote.execution.v2.FindMissingBlobsResponse\"3\x82\xd3\xe4\x93\x02-\"(/v2/{instance_name=**}/blobs:findMissing:\x01*\x12\xbc\x01\n\x10\x42\x61tchUpdateBlobs\x12\x38.build.bazel.remote.execution.v2.BatchUpdateBlobsRequest\x1a\x39.build.bazel.remote.execution.v2.BatchUpdateBlobsResponse\"3\x82\xd3\xe4\x93\x02-\"(/v2/{instance_name=**}/blobs:batchUpdate:\x01*\x12\xb4\x01\n\x0e\x42\x61tchReadBlobs\x12\x36.build.bazel.remote.execution.v2.BatchReadBlobsRequest\x1a\x37.build.bazel.remote.execution.v2.BatchReadBlobsResponse\"1\x82\xd3\xe4\x93\x02+\"&/v2/{instance_name=**}/blobs:batchRead:\x01*\x12\xc8\x01\n\x07GetTree\x12/.build.bazel.remote.execution.v2.GetTreeRequest\x1a\x30.build.bazel.remote.execution.v2.GetTreeResponse\"X\x82\xd3\xe4\x93\x02R\x12P/v2/{instance_name=**}/blobs/{root_digest.hash}/{root_digest.size_bytes}:getTree0\x01\x32\xbd\x01\n\x0c\x43\x61pabilities\x12\xac\x01\n\x0fGetCapabilities\x12\x37.build.bazel.remote.execution.v2.GetCapabilitiesRequest\x1a\x33.build.bazel.remote.execution.v2.ServerCapabilities\"+\x82\xd3\xe4\x93\x02%\x12#/v2/{instance_name=**}/capabilitiesBr\n\x1f\x62uild.bazel.remote.execution.v2B\x14RemoteExecutionProtoP\x01Z\x0fremoteexecution\xa2\x02\x03REX\xaa\x02\x1f\x42uild.Bazel.Remote.Execution.V2b\x06proto3')
 +  serialized_pb=_b('\n6build/bazel/remote/execution/v2/remote_execution.proto\x12\x1f\x62uild.bazel.remote.execution.v2\x1a\x1f\x62uild/bazel/semver/semver.proto\x1a\x1cgoogle/api/annotations.proto\x1a#google/longrunning/operations.proto\x1a\x1egoogle/protobuf/duration.proto\x1a\x1fgoogle/protobuf/timestamp.proto\x1a\x17google/rpc/status.proto\"\xd5\x01\n\x06\x41\x63tion\x12?\n\x0e\x63ommand_digest\x18\x01 \x01(\x0b\x32\'.build.bazel.remote.execution.v2.Digest\x12\x42\n\x11input_root_digest\x18\x02 \x01(\x0b\x32\'.build.bazel.remote.execution.v2.Digest\x12*\n\x07timeout\x18\x06 \x01(\x0b\x32\x19.google.protobuf.Duration\x12\x14\n\x0c\x64o_not_cache\x18\x07 \x01(\x08J\x04\x08\x03\x10\x06\"\xb7\x02\n\x07\x43ommand\x12\x11\n\targuments\x18\x01 \x03(\t\x12[\n\x15\x65nvironment_variables\x18\x02 \x03(\x0b\x32<.build.bazel.remote.execution.v2.Command.EnvironmentVariable\x12\x14\n\x0coutput_files\x18\x03 \x03(\t\x12\x1a\n\x12output_directories\x18\x04 \x03(\t\x12;\n\x08platform\x18\x05 \x01(\x0b\x32).build.bazel.remote.execution.v2.Platform\x12\x19\n\x11working_directory\x18\x06 \x01(\t\x1a\x32\n\x13\x45nvironmentVariable\x12\x0c\n\x04name\x18\x01 \x01(\t\x12\r\n\x05value\x18\x02 \x01(\t\"{\n\x08Platform\x12\x46\n\nproperties\x18\x01 \x03(\x0b\x32\x32.build.bazel.remote.execution.v2.Platform.Property\x1a\'\n\x08Property\x12\x0c\n\x04name\x18\x01 \x01(\t\x12\r\n\x05value\x18\x02 \x01(\t\"\xca\x01\n\tDirectory\x12\x38\n\x05\x66iles\x18\x01 \x03(\x0b\x32).build.bazel.remote.execution.v2.FileNode\x12\x43\n\x0b\x64irectories\x18\x02 \x03(\x0b\x32..build.bazel.remote.execution.v2.DirectoryNode\x12>\n\x08symlinks\x18\x03 \x03(\x0b\x32,.build.bazel.remote.execution.v2.SymlinkNode\"n\n\x08\x46ileNode\x12\x0c\n\x04name\x18\x01 \x01(\t\x12\x37\n\x06\x64igest\x18\x02 \x01(\x0b\x32\'.build.bazel.remote.execution.v2.Digest\x12\x15\n\ris_executable\x18\x04 \x01(\x08J\x04\x08\x03\x10\x04\"V\n\rDirectoryNode\x12\x0c\n\x04name\x18\x01 \x01(\t\x12\x37\n\x06\x64igest\x18\x02 \x01(\x0b\x32\'.build.bazel.remote.execution.v2.Digest\"+\n\x0bSymlinkNode\x12\x0c\n\x04name\x18\x01 \x01(\t\x12\x0e\n\x06target\x18\x02 \x01(\t\"*\n\x06\x44igest\x12\x0c\n\x04hash\x18\x01 \x01(\t\x12\x12\n\nsize_bytes\x18\x02 \x01(\x03\"\xec\x04\n\x16\x45xecutedActionMetadata\x12\x0e\n\x06worker\x18\x01 \x01(\t\x12\x34\n\x10queued_timestamp\x18\x02 \x01(\x0b\x32\x1a.google.protobuf.Timestamp\x12:\n\x16worker_start_timestamp\x18\x03 \x01(\x0b\x32\x1a.google.protobuf.Timestamp\x12>\n\x1aworker_completed_timestamp\x18\x04 \x01(\x0b\x32\x1a.google.protobuf.Timestamp\x12?\n\x1binput_fetch_start_timestamp\x18\x05 \x01(\x0b\x32\x1a.google.protobuf.Timestamp\x12\x43\n\x1finput_fetch_completed_timestamp\x18\x06 \x01(\x0b\x32\x1a.google.protobuf.Timestamp\x12=\n\x19\x65xecution_start_timestamp\x18\x07 \x01(\x0b\x32\x1a.google.protobuf.Timestamp\x12\x41\n\x1d\x65xecution_completed_timestamp\x18\x08 \x01(\x0b\x32\x1a.google.protobuf.Timestamp\x12\x41\n\x1doutput_upload_start_timestamp\x18\t \x01(\x0b\x32\x1a.google.protobuf.Timestamp\x12\x45\n!output_upload_completed_timestamp\x18\n \x01(\x0b\x32\x1a.google.protobuf.Timestamp\"\xd6\x04\n\x0c\x41\x63tionResult\x12\x41\n\x0coutput_files\x18\x02 \x03(\x0b\x32+.build.bazel.remote.execution.v2.OutputFile\x12L\n\x14output_file_symlinks\x18\n \x03(\x0b\x32..build.bazel.remote.execution.v2.OutputSymlink\x12L\n\x12output_directories\x18\x03 \x03(\x0b\x32\x30.build.bazel.remote.execution.v2.OutputDirectory\x12Q\n\x19output_directory_symlinks\x18\x0b \x03(\x0b\x32..build.bazel.remote.execution.v2.OutputSymlink\x12\x11\n\texit_code\x18\x04 \x01(\x05\x12\x12\n\nstdout_raw\x18\x05 \x01(\x0c\x12>\n\rstdout_digest\x18\x06 \x01(\x0b\x32\'.build.bazel.remote.execution.v2.Digest\x12\x12\n\nstderr_raw\x18\x07 \x01(\x0c\x12>\n\rstderr_digest\x18\x08 \x01(\x0b\x32\'.build.bazel.remote.execution.v2.Digest\x12S\n\x12\x65xecution_metadata\x18\t \x01(\x0b\x32\x37.build.bazel.remote.execution.v2.ExecutedActionMetadataJ\x04\x08\x01\x10\x02\"p\n\nOutputFile\x12\x0c\n\x04path\x18\x01 \x01(\t\x12\x37\n\x06\x64igest\x18\x02 \x01(\x0b\x32\'.build.bazel.remote.execution.v2.Digest\x12\x15\n\ris_executable\x18\x04 \x01(\x08J\x04\x08\x03\x10\x04\"~\n\x04Tree\x12\x38\n\x04root\x18\x01 \x01(\x0b\x32*.build.bazel.remote.execution.v2.Directory\x12<\n\x08\x63hildren\x18\x02 \x03(\x0b\x32*.build.bazel.remote.execution.v2.Directory\"c\n\x0fOutputDirectory\x12\x0c\n\x04path\x18\x01 \x01(\t\x12<\n\x0btree_digest\x18\x03 \x01(\x0b\x32\'.build.bazel.remote.execution.v2.DigestJ\x04\x08\x02\x10\x03\"-\n\rOutputSymlink\x12\x0c\n\x04path\x18\x01 \x01(\t\x12\x0e\n\x06target\x18\x02 \x01(\t\"#\n\x0f\x45xecutionPolicy\x12\x10\n\x08priority\x18\x01 \x01(\x05\"&\n\x12ResultsCachePolicy\x12\x10\n\x08priority\x18\x01 \x01(\x05\"\xb3\x02\n\x0e\x45xecuteRequest\x12\x15\n\rinstance_name\x18\x01 \x01(\t\x12\x19\n\x11skip_cache_lookup\x18\x03 \x01(\x08\x12>\n\raction_digest\x18\x06 \x01(\x0b\x32\'.build.bazel.remote.execution.v2.Digest\x12J\n\x10\x65xecution_policy\x18\x07 \x01(\x0b\x32\x30.build.bazel.remote.execution.v2.ExecutionPolicy\x12Q\n\x14results_cache_policy\x18\x08 \x01(\x0b\x32\x33.build.bazel.remote.execution.v2.ResultsCachePolicyJ\x04\x08\x02\x10\x03J\x04\x08\x04\x10\x05J\x04\x08\x05\x10\x06\"Z\n\x07LogFile\x12\x37\n\x06\x64igest\x18\x01 \x01(\x0b\x32\'.build.bazel.remote.execution.v2.Digest\x12\x16\n\x0ehuman_readable\x18\x02 \x01(\x08\"\xd0\x02\n\x0f\x45xecuteResponse\x12=\n\x06result\x18\x01 \x01(\x0b\x32-.build.bazel.remote.execution.v2.ActionResult\x12\x15\n\rcached_result\x18\x02 \x01(\x08\x12\"\n\x06status\x18\x03 \x01(\x0b\x32\x12.google.rpc.Status\x12U\n\x0bserver_logs\x18\x04 \x03(\x0b\x32@.build.bazel.remote.execution.v2.ExecuteResponse.ServerLogsEntry\x12\x0f\n\x07message\x18\x05 \x01(\t\x1a[\n\x0fServerLogsEntry\x12\x0b\n\x03key\x18\x01 \x01(\t\x12\x37\n\x05value\x18\x02 \x01(\x0b\x32(.build.bazel.remote.execution.v2.LogFile:\x02\x38\x01\"\xb3\x02\n\x18\x45xecuteOperationMetadata\x12N\n\x05stage\x18\x01 \x01(\x0e\x32?.build.bazel.remote.execution.v2.ExecuteOperationMetadata.Stage\x12>\n\raction_digest\x18\x02 \x01(\x0b\x32\'.build.bazel.remote.execution.v2.Digest\x12\x1a\n\x12stdout_stream_name\x18\x03 \x01(\t\x12\x1a\n\x12stderr_stream_name\x18\x04 \x01(\t\"O\n\x05Stage\x12\x0b\n\x07UNKNOWN\x10\x00\x12\x0f\n\x0b\x43\x41\x43HE_CHECK\x10\x01\x12\n\n\x06QUEUED\x10\x02\x12\r\n\tEXECUTING\x10\x03\x12\r\n\tCOMPLETED\x10\x04\"$\n\x14WaitExecutionRequest\x12\x0c\n\x04name\x18\x01 \x01(\t\"o\n\x16GetActionResultRequest\x12\x15\n\rinstance_name\x18\x01 \x01(\t\x12>\n\raction_digest\x18\x02 \x01(\x0b\x32\'.build.bazel.remote.execution.v2.Digest\"\x8b\x02\n\x19UpdateActionResultRequest\x12\x15\n\rinstance_name\x18\x01 \x01(\t\x12>\n\raction_digest\x18\x02 \x01(\x0b\x32\'.build.bazel.remote.execution.v2.Digest\x12\x44\n\raction_result\x18\x03 \x01(\x0b\x32-.build.bazel.remote.execution.v2.ActionResult\x12Q\n\x14results_cache_policy\x18\x04 \x01(\x0b\x32\x33.build.bazel.remote.execution.v2.ResultsCachePolicy\"o\n\x17\x46indMissingBlobsRequest\x12\x15\n\rinstance_name\x18\x01 \x01(\t\x12=\n\x0c\x62lob_digests\x18\x02 \x03(\x0b\x32\'.build.bazel.remote.execution.v2.Digest\"a\n\x18\x46indMissingBlobsResponse\x12\x45\n\x14missing_blob_digests\x18\x02 \x03(\x0b\x32\'.build.bazel.remote.execution.v2.Digest\"\xd6\x01\n\x17\x42\x61tchUpdateBlobsRequest\x12\x15\n\rinstance_name\x18\x01 \x01(\t\x12R\n\x08requests\x18\x02 \x03(\x0b\x32@.build.bazel.remote.execution.v2.BatchUpdateBlobsRequest.Request\x1aP\n\x07Request\x12\x37\n\x06\x64igest\x18\x01 \x01(\x0b\x32\'.build.bazel.remote.execution.v2.Digest\x12\x0c\n\x04\x64\x61ta\x18\x02 \x01(\x0c\"\xda\x01\n\x18\x42\x61tchUpdateBlobsResponse\x12U\n\tresponses\x18\x01 \x03(\x0b\x32\x42.build.bazel.remote.execution.v2.BatchUpdateBlobsResponse.Response\x1ag\n\x08Response\x12\x37\n\x06\x64igest\x18\x01 \x01(\x0b\x32\'.build.bazel.remote.execution.v2.Digest\x12\"\n\x06status\x18\x02 \x01(\x0b\x32\x12.google.rpc.Status\"h\n\x15\x42\x61tchReadBlobsRequest\x12\x15\n\rinstance_name\x18\x01 \x01(\t\x12\x38\n\x07\x64igests\x18\x02 \x03(\x0b\x32\'.build.bazel.remote.execution.v2.Digest\"\xe4\x01\n\x16\x42\x61tchReadBlobsResponse\x12S\n\tresponses\x18\x01 \x03(\x0b\x32@.build.bazel.remote.execution.v2.BatchReadBlobsResponse.Response\x1au\n\x08Response\x12\x37\n\x06\x64igest\x18\x01 \x01(\x0b\x32\'.build.bazel.remote.execution.v2.Digest\x12\x0c\n\x04\x64\x61ta\x18\x02 \x01(\x0c\x12\"\n\x06status\x18\x03 \x01(\x0b\x32\x12.google.rpc.Status\"\x8c\x01\n\x0eGetTreeRequest\x12\x15\n\rinstance_name\x18\x01 \x01(\t\x12<\n\x0broot_digest\x18\x02 \x01(\x0b\x32\'.build.bazel.remote.execution.v2.Digest\x12\x11\n\tpage_size\x18\x03 \x01(\x05\x12\x12\n\npage_token\x18\x04 \x01(\t\"k\n\x0fGetTreeResponse\x12?\n\x0b\x64irectories\x18\x01 \x03(\x0b\x32*.build.bazel.remote.execution.v2.Directory\x12\x17\n\x0fnext_page_token\x18\x02 \x01(\t\"/\n\x16GetCapabilitiesRequest\x12\x15\n\rinstance_name\x18\x01 \x01(\t\"\xe3\x02\n\x12ServerCapabilities\x12N\n\x12\x63\x61\x63he_capabilities\x18\x01 \x01(\x0b\x32\x32.build.bazel.remote.execution.v2.CacheCapabilities\x12V\n\x16\x65xecution_capabilities\x18\x02 \x01(\x0b\x32\x36.build.bazel.remote.execution.v2.ExecutionCapabilities\x12:\n\x16\x64\x65precated_api_version\x18\x03 \x01(\x0b\x32\x1a.build.bazel.semver.SemVer\x12\x33\n\x0flow_api_version\x18\x04 \x01(\x0b\x32\x1a.build.bazel.semver.SemVer\x12\x34\n\x10high_api_version\x18\x05 \x01(\x0b\x32\x1a.build.bazel.semver.SemVer\"7\n\x1d\x41\x63tionCacheUpdateCapabilities\x12\x16\n\x0eupdate_enabled\x18\x01 \x01(\x08\"\xac\x01\n\x14PriorityCapabilities\x12W\n\npriorities\x18\x01 \x03(\x0b\x32\x43.build.bazel.remote.execution.v2.PriorityCapabilities.PriorityRange\x1a;\n\rPriorityRange\x12\x14\n\x0cmin_priority\x18\x01 \x01(\x05\x12\x14\n\x0cmax_priority\x18\x02 \x01(\x05\"\x88\x04\n\x11\x43\x61\x63heCapabilities\x12H\n\x0f\x64igest_function\x18\x01 \x03(\x0e\x32/.build.bazel.remote.execution.v2.DigestFunction\x12h\n action_cache_update_capabilities\x18\x02 \x01(\x0b\x32>.build.bazel.remote.execution.v2.ActionCacheUpdateCapabilities\x12Z\n\x1b\x63\x61\x63he_priority_capabilities\x18\x03 \x01(\x0b\x32\x35.build.bazel.remote.execution.v2.PriorityCapabilities\x12\"\n\x1amax_batch_total_size_bytes\x18\x04 \x01(\x03\x12v\n\x1esymlink_absolute_path_strategy\x18\x05 \x01(\x0e\x32N.build.bazel.remote.execution.v2.CacheCapabilities.SymlinkAbsolutePathStrategy\"G\n\x1bSymlinkAbsolutePathStrategy\x12\x0b\n\x07UNKNOWN\x10\x00\x12\x0e\n\nDISALLOWED\x10\x01\x12\x0b\n\x07\x41LLOWED\x10\x02\"\xd7\x01\n\x15\x45xecutionCapabilities\x12H\n\x0f\x64igest_function\x18\x01 \x01(\x0e\x32/.build.bazel.remote.execution.v2.DigestFunction\x12\x14\n\x0c\x65xec_enabled\x18\x02 \x01(\x08\x12^\n\x1f\x65xecution_priority_capabilities\x18\x03 \x01(\x0b\x32\x35.build.bazel.remote.execution.v2.PriorityCapabilities\"6\n\x0bToolDetails\x12\x11\n\ttool_name\x18\x01 \x01(\t\x12\x14\n\x0ctool_version\x18\x02 \x01(\t\"\xa7\x01\n\x0fRequestMetadata\x12\x42\n\x0ctool_details\x18\x01 \x01(\x0b\x32,.build.bazel.remote.execution.v2.ToolDetails\x12\x11\n\taction_id\x18\x02 \x01(\t\x12\x1a\n\x12tool_invocation_id\x18\x03 \x01(\t\x12!\n\x19\x63orrelated_invocations_id\x18\x04 \x01(\t*<\n\x0e\x44igestFunction\x12\x0b\n\x07UNKNOWN\x10\x00\x12\n\n\x06SHA256\x10\x01\x12\x08\n\x04SHA1\x10\x02\x12\x07\n\x03MD5\x10\x03\x32\xb9\x02\n\tExecution\x12\x8e\x01\n\x07\x45xecute\x12/.build.bazel.remote.execution.v2.ExecuteRequest\x1a\x1d.google.longrunning.Operation\"1\x82\xd3\xe4\x93\x02+\"&/v2/{instance_name=**}/actions:execute:\x01*0\x01\x12\x9a\x01\n\rWaitExecution\x12\x35.build.bazel.remote.execution.v2.WaitExecutionRequest\x1a\x1d.google.longrunning.Operation\"1\x82\xd3\xe4\x93\x02+\"&/v2/{name=operations/**}:waitExecution:\x01*0\x01\x32\xd6\x03\n\x0b\x41\x63tionCache\x12\xd7\x01\n\x0fGetActionResult\x12\x37.build.bazel.remote.execution.v2.GetActionResultRequest\x1a-.build.bazel.remote.execution.v2.ActionResult\"\\\x82\xd3\xe4\x93\x02V\x12T/v2/{instance_name=**}/actionResults/{action_digest.hash}/{action_digest.size_bytes}\x12\xec\x01\n\x12UpdateActionResult\x12:.build.bazel.remote.execution.v2.UpdateActionResultRequest\x1a-.build.bazel.remote.execution.v2.ActionResult\"k\x82\xd3\xe4\x93\x02\x65\x1aT/v2/{instance_name=**}/actionResults/{action_digest.hash}/{action_digest.size_bytes}:\raction_result2\x9b\x06\n\x19\x43ontentAddressableStorage\x12\xbc\x01\n\x10\x46indMissingBlobs\x12\x38.build.bazel.remote.execution.v2.FindMissingBlobsRequest\x1a\x39.build.bazel.remote.execution.v2.FindMissingBlobsResponse\"3\x82\xd3\xe4\x93\x02-\"(/v2/{instance_name=**}/blobs:findMissing:\x01*\x12\xbc\x01\n\x10\x42\x61tchUpdateBlobs\x12\x38.build.bazel.remote.execution.v2.BatchUpdateBlobsRequest\x1a\x39.build.bazel.remote.execution.v2.BatchUpdateBlobsResponse\"3\x82\xd3\xe4\x93\x02-\"(/v2/{instance_name=**}/blobs:batchUpdate:\x01*\x12\xb4\x01\n\x0e\x42\x61tchReadBlobs\x12\x36.build.bazel.remote.execution.v2.BatchReadBlobsRequest\x1a\x37.build.bazel.remote.execution.v2.BatchReadBlobsResponse\"1\x82\xd3\xe4\x93\x02+\"&/v2/{instance_name=**}/blobs:batchRead:\x01*\x12\xc8\x01\n\x07GetTree\x12/.build.bazel.remote.execution.v2.GetTreeRequest\x1a\x30.build.bazel.remote.execution.v2.GetTreeResponse\"X\x82\xd3\xe4\x93\x02R\x12P/v2/{instance_name=**}/blobs/{root_digest.hash}/{root_digest.size_bytes}:getTree0\x01\x32\xbd\x01\n\x0c\x43\x61pabilities\x12\xac\x01\n\x0fGetCapabilities\x12\x37.build.bazel.remote.execution.v2.GetCapabilitiesRequest\x1a\x33.build.bazel.remote.execution.v2.ServerCapabilities\"+\x82\xd3\xe4\x93\x02%\x12#/v2/{instance_name=**}/capabilitiesBr\n\x1f\x62uild.bazel.remote.execution.v2B\x14RemoteExecutionProtoP\x01Z\x0fremoteexecution\xa2\x02\x03REX\xaa\x02\x1f\x42uild.Bazel.Remote.Execution.V2b\x06proto3')
+   ,
    dependencies=[build_dot_bazel_dot_semver_dot_semver__pb2.DESCRIPTOR,google_dot_api_dot_annotations__pb2.DESCRIPTOR,google_dot_longrunning_dot_operations__pb2.DESCRIPTOR,google_dot_protobuf_dot_duration__pb2.DESCRIPTOR,google_dot_protobuf_dot_timestamp__pb2.DESCRIPTOR,google_dot_rpc_dot_status__pb2.DESCRIPTOR,])
@@ -55,8 +55,8 @@ _DIGESTFUNCTION = _descriptor.EnumDescriptor(
    ],
    containing_type=None,
    serialized_options=None,
 -  serialized_start=7421,
 -  serialized_end=7481,
 +  serialized_start=7438,
 +  serialized_end=7498,
+ )
  _sym_db.RegisterEnumDescriptor(_DIGESTFUNCTION)
@@ -96,8 +96,8 @@ _EXECUTEOPERATIONMETADATA_STAGE = _descriptor.EnumDescriptor(
    ],
    containing_type=None,
    serialized_options=None,
 -  serialized_start=4074,
 -  serialized_end=4153,
 +  serialized_start=4091,
 +  serialized_end=4170,
+ )
  _sym_db.RegisterEnumDescriptor(_EXECUTEOPERATIONMETADATA_STAGE)
@@ -122,8 +122,8 @@ _CACHECAPABILITIES_SYMLINKABSOLUTEPATHSTRATEGY = _descriptor.EnumDescriptor(
    ],
    containing_type=None,
    serialized_options=None,
 -  serialized_start=6904,
 -  serialized_end=6975,
 +  serialized_start=6921,
 +  serialized_end=6992,
+ )
  _sym_db.RegisterEnumDescriptor(_CACHECAPABILITIES_SYMLINKABSOLUTEPATHSTRATEGY)
@@ -1094,8 +1094,8 @@ _EXECUTERESPONSE_SERVERLOGSENTRY = _descriptor.Descriptor(
    extension_ranges=[],
    oneofs=[
    ],
 -  serialized_start=3752,
 -  serialized_end=3843,
 +  serialized_start=3769,
 +  serialized_end=3860,
+ )
  _EXECUTERESPONSE = _descriptor.Descriptor(
@@ -1133,6 +1133,13 @@ _EXECUTERESPONSE = _descriptor.Descriptor(
        message_type=None, enum_type=None, containing_type=None,
        is_extension=False, extension_scope=None,
        serialized_options=None, file=DESCRIPTOR),
 +    _descriptor.FieldDescriptor(
 +      name='message', full_name='build.bazel.remote.execution.v2.ExecuteResponse.message', index=4,
 +      number=5, type=9, cpp_type=9, label=1,
 +      has_default_value=False, default_value=_b("").decode('utf-8'),
 +      message_type=None, enum_type=None, containing_type=None,
 +      is_extension=False, extension_scope=None,
 +      serialized_options=None, file=DESCRIPTOR),
    ],
    extensions=[
    ],
@@ -1146,7 +1153,7 @@ _EXECUTERESPONSE = _descriptor.Descriptor(
    oneofs=[
    ],
    serialized_start=3524,
 -  serialized_end=3843,
 +  serialized_end=3860,
+ )
@@ -1198,8 +1205,8 @@ _EXECUTEOPERATIONMETADATA = _descriptor.Descriptor(
    extension_ranges=[],
    oneofs=[
    ],
 -  serialized_start=3846,
 -  serialized_end=4153,
 +  serialized_start=3863,
 +  serialized_end=4170,
+ )
@@ -1229,8 +1236,8 @@ _WAITEXECUTIONREQUEST = _descriptor.Descriptor(
    extension_ranges=[],
    oneofs=[
    ],
 -  serialized_start=4155,
 -  serialized_end=4191,
 +  serialized_start=4172,
 +  serialized_end=4208,
+ )
@@ -1267,8 +1274,8 @@ _GETACTIONRESULTREQUEST = _descriptor.Descriptor(
    extension_ranges=[],
    oneofs=[
    ],
 -  serialized_start=4193,
 -  serialized_end=4304,
 +  serialized_start=4210,
 +  serialized_end=4321,
+ )
@@ -1319,8 +1326,8 @@ _UPDATEACTIONRESULTREQUEST = _descriptor.Descriptor(
    extension_ranges=[],
    oneofs=[
    ],
 -  serialized_start=4307,
 -  serialized_end=4574,
 +  serialized_start=4324,
 +  serialized_end=4591,
+ )
@@ -1357,8 +1364,8 @@ _FINDMISSINGBLOBSREQUEST = _descriptor.Descriptor(
    extension_ranges=[],
    oneofs=[
    ],
 -  serialized_start=4576,
 -  serialized_end=4687,
 +  serialized_start=4593,
 +  serialized_end=4704,
+ )
@@ -1388,8 +1395,8 @@ _FINDMISSINGBLOBSRESPONSE = _descriptor.Descriptor(
    extension_ranges=[],
    oneofs=[
    ],
 -  serialized_start=4689,
 -  serialized_end=4786,
 +  serialized_start=4706,
 +  serialized_end=4803,
+ )
@@ -1426,8 +1433,8 @@ _BATCHUPDATEBLOBSREQUEST_REQUEST = _descriptor.Descriptor(
    extension_ranges=[],
    oneofs=[
    ],
 -  serialized_start=4923,
 -  serialized_end=5003,
 +  serialized_start=4940,
 +  serialized_end=5020,
+ )
  _BATCHUPDATEBLOBSREQUEST = _descriptor.Descriptor(
@@ -1463,8 +1470,8 @@ _BATCHUPDATEBLOBSREQUEST = _descriptor.Descriptor(
    extension_ranges=[],
    oneofs=[
    ],
 -  serialized_start=4789,
 -  serialized_end=5003,
 +  serialized_start=4806,
 +  serialized_end=5020,
+ )
@@ -1501,8 +1508,8 @@ _BATCHUPDATEBLOBSRESPONSE_RESPONSE = _descriptor.Descriptor(
    extension_ranges=[],
    oneofs=[
    ],
 -  serialized_start=5121,
 -  serialized_end=5224,
 +  serialized_start=5138,
 +  serialized_end=5241,
+ )
  _BATCHUPDATEBLOBSRESPONSE = _descriptor.Descriptor(
@@ -1531,8 +1538,8 @@ _BATCHUPDATEBLOBSRESPONSE = _descriptor.Descriptor(
    extension_ranges=[],
    oneofs=[
    ],
 -  serialized_start=5006,
 -  serialized_end=5224,
 +  serialized_start=5023,
 +  serialized_end=5241,
+ )
@@ -1569,8 +1576,8 @@ _BATCHREADBLOBSREQUEST = _descriptor.Descriptor(
    extension_ranges=[],
    oneofs=[
    ],
 -  serialized_start=5226,
 -  serialized_end=5330,
 +  serialized_start=5243,
 +  serialized_end=5347,
+ )
@@ -1614,8 +1621,8 @@ _BATCHREADBLOBSRESPONSE_RESPONSE = _descriptor.Descriptor(
    extension_ranges=[],
    oneofs=[
    ],
 -  serialized_start=5444,
 -  serialized_end=5561,
 +  serialized_start=5461,
 +  serialized_end=5578,
+ )
  _BATCHREADBLOBSRESPONSE = _descriptor.Descriptor(
@@ -1644,8 +1651,8 @@ _BATCHREADBLOBSRESPONSE = _descriptor.Descriptor(
    extension_ranges=[],
    oneofs=[
    ],
 -  serialized_start=5333,
 -  serialized_end=5561,
 +  serialized_start=5350,
 +  serialized_end=5578,
+ )
@@ -1696,8 +1703,8 @@ _GETTREEREQUEST = _descriptor.Descriptor(
    extension_ranges=[],
    oneofs=[
    ],
 -  serialized_start=5564,
 -  serialized_end=5704,
 +  serialized_start=5581,
 +  serialized_end=5721,
+ )
@@ -1734,8 +1741,8 @@ _GETTREERESPONSE = _descriptor.Descriptor(
    extension_ranges=[],
    oneofs=[
    ],
 -  serialized_start=5706,
 -  serialized_end=5813,
 +  serialized_start=5723,
 +  serialized_end=5830,
+ )
@@ -1765,8 +1772,8 @@ _GETCAPABILITIESREQUEST = _descriptor.Descriptor(
    extension_ranges=[],
    oneofs=[
    ],
 -  serialized_start=5815,
 -  serialized_end=5862,
 +  serialized_start=5832,
 +  serialized_end=5879,
+ )
@@ -1824,8 +1831,8 @@ _SERVERCAPABILITIES = _descriptor.Descriptor(
    extension_ranges=[],
    oneofs=[
    ],
 -  serialized_start=5865,
 -  serialized_end=6220,
 +  serialized_start=5882,
 +  serialized_end=6237,
+ )
@@ -1855,8 +1862,8 @@ _ACTIONCACHEUPDATECAPABILITIES = _descriptor.Descriptor(
    extension_ranges=[],
    oneofs=[
    ],
 -  serialized_start=6222,
 -  serialized_end=6277,
 +  serialized_start=6239,
 +  serialized_end=6294,
+ )
@@ -1893,8 +1900,8 @@ _PRIORITYCAPABILITIES_PRIORITYRANGE = _descriptor.Descriptor(
    extension_ranges=[],
    oneofs=[
    ],
 -  serialized_start=6393,
 -  serialized_end=6452,
 +  serialized_start=6410,
 +  serialized_end=6469,
+ )
  _PRIORITYCAPABILITIES = _descriptor.Descriptor(
@@ -1923,8 +1930,8 @@ _PRIORITYCAPABILITIES = _descriptor.Descriptor(
    extension_ranges=[],
    oneofs=[
    ],
 -  serialized_start=6280,
 -  serialized_end=6452,
 +  serialized_start=6297,
 +  serialized_end=6469,
+ )
@@ -1983,8 +1990,8 @@ _CACHECAPABILITIES = _descriptor.Descriptor(
    extension_ranges=[],
    oneofs=[
    ],
 -  serialized_start=6455,
 -  serialized_end=6975,
 +  serialized_start=6472,
 +  serialized_end=6992,
+ )
@@ -2028,8 +2035,8 @@ _EXECUTIONCAPABILITIES = _descriptor.Descriptor(
    extension_ranges=[],
    oneofs=[
    ],
 -  serialized_start=6978,
 -  serialized_end=7193,
 +  serialized_start=6995,
 +  serialized_end=7210,
+ )
@@ -2066,8 +2073,8 @@ _TOOLDETAILS = _descriptor.Descriptor(
    extension_ranges=[],
    oneofs=[
    ],
 -  serialized_start=7195,
 -  serialized_end=7249,
 +  serialized_start=7212,
 +  serialized_end=7266,
+ )
@@ -2118,8 +2125,8 @@ _REQUESTMETADATA = _descriptor.Descriptor(
    extension_ranges=[],
    oneofs=[
    ],
 -  serialized_start=7252,
 -  serialized_end=7419,
 +  serialized_start=7269,
 +  serialized_end=7436,
+ )
  _ACTION.fields_by_name['command_digest'].message_type = _DIGEST
@@ -2583,8 +2590,8 @@ _EXECUTION = _descriptor.ServiceDescriptor(
    file=DESCRIPTOR,
    index=0,
    serialized_options=None,
 -  serialized_start=7484,
 -  serialized_end=7797,
 +  serialized_start=7501,
 +  serialized_end=7814,
    methods=[
    _descriptor.MethodDescriptor(
      name='Execute',
@@ -2616,8 +2623,8 @@ _ACTIONCACHE = _descriptor.ServiceDescriptor(
    file=DESCRIPTOR,
    index=1,
    serialized_options=None,
 -  serialized_start=7800,
 -  serialized_end=8270,
 +  serialized_start=7817,
 +  serialized_end=8287,
    methods=[
    _descriptor.MethodDescriptor(
      name='GetActionResult',
@@ -2649,8 +2656,8 @@ _CONTENTADDRESSABLESTORAGE = _descriptor.ServiceDescriptor(
    file=DESCRIPTOR,
    index=2,
    serialized_options=None,
 -  serialized_start=8273,
 -  serialized_end=9068,
 +  serialized_start=8290,
 +  serialized_end=9085,
    methods=[
    _descriptor.MethodDescriptor(
      name='FindMissingBlobs',
@@ -2700,8 +2707,8 @@ _CAPABILITIES = _descriptor.ServiceDescriptor(
    file=DESCRIPTOR,
    index=3,
    serialized_options=None,
 -  serialized_start=9071,
 -  serialized_end=9260,
 +  serialized_start=9088,
 +  serialized_end=9277,
    methods=[
    _descriptor.MethodDescriptor(
      name='GetCapabilities',

buildgrid/_protos/build/bazel/remote/execution/v2/remote_execution_pb2_grpc.py

@@ -87,6 +87,7 @@ class ExecutionServicer(object):
      action will be reported in the `status` field of the `ExecuteResponse`. The
      server MUST NOT set the `error` field of the `Operation` proto.
      The possible errors include:
++
      * `INVALID_ARGUMENT`: One or more arguments are invalid.
      * `FAILED_PRECONDITION`: One or more errors occurred in setting up the
      action requested, such as a missing input or command or no worker being
@@ -210,6 +211,7 @@ class ActionCacheServicer(object):
      """Retrieve a cached execution result.
      Errors:
++
      * `NOT_FOUND`: The requested `ActionResult` is not in the cache.
      """
      context.set_code(grpc.StatusCode.UNIMPLEMENTED)
@@ -219,11 +221,6 @@ class ActionCacheServicer(object):
    def UpdateActionResult(self, request, context):
      """Upload a new execution result.
 -    This method is intended for servers which implement the distributed cache
 -    independently of the
 -    [Execution][build.bazel.remote.execution.v2.Execution] API. As a
 -    result, it is OPTIONAL for servers to implement.
+-
      In order to allow the server to perform access control based on the type of
      action, and to assist with client debugging, the client MUST first upload
      the [Action][build.bazel.remote.execution.v2.Execution] that produced the
@@ -232,7 +229,10 @@ class ActionCacheServicer(object):
      `ContentAddressableStorage`.
      Errors:
 -    * `UNIMPLEMENTED`: This method is not supported by the server.
++
 +    * `INVALID_ARGUMENT`: One or more arguments are invalid.
 +    * `FAILED_PRECONDITION`: One or more errors occurred in updating the
 +    action result, such as a missing command or action.
      * `RESOURCE_EXHAUSTED`: There is insufficient storage space to add the
      entry to the cache.
      """
@@ -299,6 +299,9 @@ class ContentAddressableStorageStub(object):
    by the server. For servers which do not support multiple instances, then the
    `instance_name` is the empty path and the leading slash is omitted, so that
    the `resource_name` becomes `uploads/{uuid}/blobs/{hash}/{size}`.
 +  To simplify parsing, a path segment cannot equal any of the following
 +  keywords: `blobs`, `uploads`, `actions`, `actionResults`, `operations` and
 +  `capabilities`.
    When attempting an upload, if another client has already completed the upload
    (which may occur in the middle of a single upload if another client uploads
@@ -395,6 +398,9 @@ class ContentAddressableStorageServicer(object):
    by the server. For servers which do not support multiple instances, then the
    `instance_name` is the empty path and the leading slash is omitted, so that
    the `resource_name` becomes `uploads/{uuid}/blobs/{hash}/{size}`.
 +  To simplify parsing, a path segment cannot equal any of the following
 +  keywords: `blobs`, `uploads`, `actions`, `actionResults`, `operations` and
 +  `capabilities`.
    When attempting an upload, if another client has already completed the upload
    (which may occur in the middle of a single upload if another client uploads
@@ -450,10 +456,12 @@ class ContentAddressableStorageServicer(object):
      independently.
      Errors:
++
      * `INVALID_ARGUMENT`: The client attempted to upload more than the
      server supported limit.
      Individual requests may return the following errors, additionally:
++
      * `RESOURCE_EXHAUSTED`: There is insufficient disk quota to store the blob.
      * `INVALID_ARGUMENT`: The
      [Digest][build.bazel.remote.execution.v2.Digest] does not match the
@@ -478,6 +486,7 @@ class ContentAddressableStorageServicer(object):
      independently.
      Errors:
++
      * `INVALID_ARGUMENT`: The client attempted to read more than the
      server supported limit.

buildgrid/_protos/build/bazel/semver/semver.proto

@@ -23,9 +23,19 @@ option java_outer_classname = "SemverProto";
  option java_package = "build.bazel.semver";
  option objc_class_prefix = "SMV";
 +// The full version of a given tool.
  message SemVer {
 +  // The major version, e.g 10 for 10.2.3.
    int32 major = 1;
++
 +  // The minor version, e.g. 2 for 10.2.3.
    int32 minor = 2;
++
 +  // The patch version, e.g 3 for 10.2.3.
    int32 patch = 3;
++
 +  // The pre-release version. Either this field or major/minor/patch fields
 +  // must be filled. They are mutually exclusive. Pre-release versions are
 +  // assumed to be earlier than any released versions.
    string prerelease = 4;
+ }

buildgrid/client/cas.py

@@ -24,11 +24,10 @@ from buildgrid._exceptions import NotFoundError
  from buildgrid._protos.build.bazel.remote.execution.v2 import remote_execution_pb2, remote_execution_pb2_grpc
  from buildgrid._protos.google.bytestream import bytestream_pb2, bytestream_pb2_grpc
  from buildgrid._protos.google.rpc import code_pb2
 -from buildgrid.settings import HASH, MAX_REQUEST_SIZE, MAX_REQUEST_COUNT
 +from buildgrid.client.capabilities import CapabilitiesInterface
 +from buildgrid.settings import HASH, MAX_REQUEST_SIZE, MAX_REQUEST_COUNT, BATCH_REQUEST_SIZE_THRESHOLD
  from buildgrid.utils import create_digest, merkle_tree_maker
 -# Maximum size for a queueable file:
 -FILE_SIZE_THRESHOLD = 1 * 1024 * 1024
  _FileRequest = namedtuple('FileRequest', ['digest', 'output_paths'])
@@ -50,6 +49,67 @@ class _CallCache:
          return name in cls.__calls[channel]
 +class _CasBatchRequestSizesCache:
 +    """Cache that stores, for each remote, the limit of bytes that can
 +    be transferred using batches as well as a threshold that determines
 +    when a file can be fetched as part of a batch request.
 +    """
 +    __cas_max_batch_transfer_size = {}
 +    __cas_batch_request_size_threshold = {}
++
 +    @classmethod
 +    def max_effective_batch_size_bytes(cls, channel, instance_name):
 +        """Returns the maximum number of bytes that can be transferred
 +        using batch methods for the given remote.
 +        """
 +        if channel not in cls.__cas_max_batch_transfer_size:
 +            cls.__cas_max_batch_transfer_size[channel] = dict()
++
 +        if instance_name not in cls.__cas_max_batch_transfer_size[channel]:
 +            max_batch_size = cls._get_server_max_batch_total_size_bytes(channel,
 +                                                                        instance_name)
++
 +            cls.__cas_max_batch_transfer_size[channel][instance_name] = max_batch_size
++
 +        return cls.__cas_max_batch_transfer_size[channel][instance_name]
++
 +    @classmethod
 +    def batch_request_size_threshold(cls, channel, instance_name):
 +        if channel not in cls.__cas_batch_request_size_threshold:
 +            cls.__cas_batch_request_size_threshold[channel] = dict()
++
 +        if instance_name not in cls.__cas_batch_request_size_threshold[channel]:
 +            # Computing the threshold:
 +            max_batch_size = cls.max_effective_batch_size_bytes(channel,
 +                                                                instance_name)
 +            threshold = BATCH_REQUEST_SIZE_THRESHOLD * max_batch_size
++
 +            cls.__cas_batch_request_size_threshold[channel][instance_name] = threshold
++
 +        return cls.__cas_batch_request_size_threshold[channel][instance_name]
++
 +    @classmethod
 +    def _get_server_max_batch_total_size_bytes(cls, channel, instance_name):
 +        """Returns the maximum number of bytes that can be effectively
 +        transferred using batches, considering the limits imposed by
 +        the server's configuration and by gRPC.
 +        """
 +        try:
 +            capabilities_interface = CapabilitiesInterface(channel)
 +            server_capabilities = capabilities_interface.get_capabilities(instance_name)
++
 +            cache_capabilities = server_capabilities.cache_capabilities
++
 +            max_batch_total_size = cache_capabilities.max_batch_total_size_bytes
 +            # The server could set this value to 0 (no limit set).
 +            if max_batch_total_size:
 +                return min(max_batch_total_size, MAX_REQUEST_SIZE)
 +        except Exception:
 +            pass
++
 +        return MAX_REQUEST_SIZE
++
++
  @contextmanager
  def download(channel, instance=None, u_uid=None):
      """Context manager generator for the :class:`Downloader` class."""
@@ -189,7 +249,7 @@ class Downloader:
          if not os.path.isabs(file_path):
              file_path = os.path.abspath(file_path)
 -        if not queue or digest.size_bytes > FILE_SIZE_THRESHOLD:
 +        if not queue or digest.size_bytes > self._queueable_file_size_threshold():
              self._fetch_file(digest, file_path, is_executable=is_executable)
          else:
              self._queue_file(digest, file_path, is_executable=is_executable)
@@ -334,9 +394,11 @@ class Downloader:
      def _queue_file(self, digest, file_path, is_executable=False):
          """Queues a file for later batch download"""
 -        if self.__file_request_size + digest.ByteSize() > MAX_REQUEST_SIZE:
 +        batch_size_limit = self._max_effective_batch_size_bytes()
++
 +        if self.__file_request_size + digest.ByteSize() > batch_size_limit:
              self.flush()
 -        elif self.__file_response_size + digest.size_bytes > MAX_REQUEST_SIZE:
 +        elif self.__file_response_size + digest.size_bytes > batch_size_limit:
              self.flush()
          elif self.__file_request_count >= MAX_REQUEST_COUNT:
              self.flush()
@@ -498,6 +560,20 @@ class Downloader:
              os.symlink(symlink_path, target_path)
 +    def _max_effective_batch_size_bytes(self):
 +        """Returns the effective maximum number of bytes that can be
 +        transferred using batches, considering gRPC maximum message size.
 +        """
 +        return _CasBatchRequestSizesCache.max_effective_batch_size_bytes(self.channel,
 +                                                                         self.instance_name)
++
 +    def _queueable_file_size_threshold(self):
 +        """Returns the size limit up until which files can be queued to
 +        be requested in a batch.
 +        """
 +        return _CasBatchRequestSizesCache.batch_request_size_threshold(self.channel,
 +                                                                       self.instance_name)
++
  @contextmanager
  def upload(channel, instance=None, u_uid=None):
@@ -563,7 +639,8 @@ class Uploader:
          Returns:
              :obj:`Digest`: the sent blob's digest.
          """
 -        if not queue or len(blob) > FILE_SIZE_THRESHOLD:
++
 +        if not queue or len(blob) > self._queueable_file_size_threshold():
              blob_digest = self._send_blob(blob, digest=digest)
          else:
              blob_digest = self._queue_blob(blob, digest=digest)
@@ -589,7 +666,7 @@ class Uploader:
          """
          message_blob = message.SerializeToString()
 -        if not queue or len(message_blob) > FILE_SIZE_THRESHOLD:
 +        if not queue or len(message_blob) > self._queueable_file_size_threshold():
              message_digest = self._send_blob(message_blob, digest=digest)
          else:
              message_digest = self._queue_blob(message_blob, digest=digest)
@@ -622,7 +699,7 @@ class Uploader:
          with open(file_path, 'rb') as bytes_steam:
              file_bytes = bytes_steam.read()
 -        if not queue or len(file_bytes) > FILE_SIZE_THRESHOLD:
 +        if not queue or len(file_bytes) > self._queueable_file_size_threshold():
              file_digest = self._send_blob(file_bytes)
          else:
              file_digest = self._queue_blob(file_bytes)
@@ -795,7 +872,12 @@ class Uploader:
              blob_digest.hash = HASH(blob).hexdigest()
              blob_digest.size_bytes = len(blob)
 -        if self.__request_size + blob_digest.size_bytes > MAX_REQUEST_SIZE:
 +        # If we are here queueing a file we know that its size is
 +        # smaller than gRPC's message size limit.
 +        # We'll make a single batch request as big as the server allows.
 +        batch_size_limit = self._max_effective_batch_size_bytes()
++
 +        if self.__request_size + blob_digest.size_bytes > batch_size_limit:
              self.flush()
          elif self.__request_count >= MAX_REQUEST_COUNT:
              self.flush()
@@ -851,3 +933,17 @@ class Uploader:
                  written_digests.append(self._send_blob(blob, digest=digest))
          return written_digests
++
 +    def _max_effective_batch_size_bytes(self):
 +        """Returns the effective maximum number of bytes that can be
 +        transferred using batches, considering gRPC maximum message size.
 +        """
 +        return _CasBatchRequestSizesCache.max_effective_batch_size_bytes(self.channel,
 +                                                                         self.instance_name)
++
 +    def _queueable_file_size_threshold(self):
 +        """Returns the size limit up until which files can be queued to
 +        be requested in a batch.
 +        """
 +        return _CasBatchRequestSizesCache.batch_request_size_threshold(self.channel,
 +                                                                       self.instance_name)

buildgrid/server/actioncache/storage.py

@@ -40,12 +40,25 @@ class ActionCache(ReferenceCache):
          self.__logger = logging.getLogger(__name__)
 +        self._instance_name = None
++
          self._cache_failed_actions = cache_failed_actions
      # --- Public API ---
 +    @property
 +    def instance_name(self):
 +        return self._instance_name
++
      def register_instance_with_server(self, instance_name, server):
 -        server.add_action_cache_instance(self, instance_name)
 +        """Names and registers the action-cache instance with a given server."""
 +        if self._instance_name is None:
 +            server.add_action_cache_instance(self, instance_name)
++
 +            self._instance_name = instance_name
++
 +        else:
 +            raise AssertionError("Instance already registered")
      def get_action_result(self, action_digest):
          """Retrieves the cached result for an action."""

buildgrid/server/bots/instance.py

@@ -33,16 +33,31 @@ class BotsInterface:
      def __init__(self, scheduler):
          self.__logger = logging.getLogger(__name__)
 +        self._scheduler = scheduler
 +        self._instance_name = None
++
          self._bot_ids = {}
          self._assigned_leases = {}
 -        self._scheduler = scheduler
++
 +    # --- Public API ---
++
 +    @property
 +    def instance_name(self):
 +        return self._instance_name
      @property
      def scheduler(self):
          return self._scheduler
      def register_instance_with_server(self, instance_name, server):
 -        server.add_bots_interface(self, instance_name)
 +        """Names and registers the bots interface with a given server."""
 +        if self._instance_name is None:
 +            server.add_bots_interface(self, instance_name)
++
 +            self._instance_name = instance_name
++
 +        else:
 +            raise AssertionError("Instance already registered")
      def create_bot_session(self, parent, bot_session):
          """ Creates a new bot session. Server should assign a unique
@@ -98,6 +113,8 @@ class BotsInterface:
          self._request_leases(bot_session)
          return bot_session
 +    # --- Private API ---
++
      def _request_leases(self, bot_session):
          # Only send one lease at a time currently.
          if not bot_session.leases:

buildgrid/server/capabilities/instance.py

@@ -16,18 +16,39 @@
  import logging
  from buildgrid._protos.build.bazel.remote.execution.v2 import remote_execution_pb2
 +from buildgrid._protos.build.bazel.semver import semver_pb2
 +from buildgrid.settings import HIGH_REAPI_VERSION, LOW_REAPI_VERSION
  class CapabilitiesInstance:
      def __init__(self, cas_instance=None, action_cache_instance=None, execution_instance=None):
          self.__logger = logging.getLogger(__name__)
++
 +        self._instance_name = None
++
          self.__cas_instance = cas_instance
          self.__action_cache_instance = action_cache_instance
          self.__execution_instance = execution_instance
 +        self.__high_api_version = None
 +        self.__low_api_version = None
++
 +    # --- Public API ---
++
 +    @property
 +    def instance_name(self):
 +        return self._instance_name
++
      def register_instance_with_server(self, instance_name, server):
 -        server.add_capabilities_instance(self, instance_name)
 +        """Names and registers the capabilities instance with a given server."""
 +        if self._instance_name is None:
 +            server.add_capabilities_instance(self, instance_name)
++
 +            self._instance_name = instance_name
++
 +        else:
 +            raise AssertionError("Instance already registered")
      def add_cas_instance(self, cas_instance):
          self.__cas_instance = cas_instance
@@ -39,17 +60,24 @@ class CapabilitiesInstance:
          self.__execution_instance = execution_instance
      def get_capabilities(self):
 +        cache_capabilities = self._get_cache_capabilities()
 +        execution_capabilities = self._get_capabilities_execution()
++
 +        if self.__high_api_version is None:
 +            self.__high_api_version = self._split_semantic_version(HIGH_REAPI_VERSION)
 +        if self.__low_api_version is None:
 +            self.__low_api_version = self._split_semantic_version(LOW_REAPI_VERSION)
++
          server_capabilities = remote_execution_pb2.ServerCapabilities()
 -        server_capabilities.cache_capabilities.CopyFrom(self._get_cache_capabilities())
 -        server_capabilities.execution_capabilities.CopyFrom(self._get_capabilities_execution())
 -        # TODO
 -        # When API is stable, fill out SemVer values
 -        # server_capabilities.deprecated_api_version =
 -        # server_capabilities.low_api_version =
 -        # server_capabilities.low_api_version =
 -        # server_capabilities.hig_api_version =
 +        server_capabilities.cache_capabilities.CopyFrom(cache_capabilities)
 +        server_capabilities.execution_capabilities.CopyFrom(execution_capabilities)
 +        server_capabilities.low_api_version.CopyFrom(self.__low_api_version)
 +        server_capabilities.high_api_version.CopyFrom(self.__high_api_version)
++
          return server_capabilities
 +    # --- Private API ---
++
      def _get_cache_capabilities(self):
          capabilities = remote_execution_pb2.CacheCapabilities()
          action_cache_update_capabilities = remote_execution_pb2.ActionCacheUpdateCapabilities()
@@ -79,3 +107,13 @@ class CapabilitiesInstance:
              capabilities.exec_enabled = False
          return capabilities
++
 +    def _split_semantic_version(self, version_string):
 +        major_version, minor_version, patch_version = version_string.split('.')
++
 +        semantic_version = semver_pb2.SemVer()
 +        semantic_version.major = int(major_version)
 +        semantic_version.minor = int(minor_version)
 +        semantic_version.patch = int(patch_version)
++
 +        return semantic_version

buildgrid/server/capabilities/service.py

@@ -49,8 +49,17 @@ class CapabilitiesService(remote_execution_pb2_grpc.CapabilitiesServicer):
      @authorize(AuthContext)
      def GetCapabilities(self, request, context):
 +        """Handles GetCapabilitiesRequest messages.
++
 +        Args:
 +            request (GetCapabilitiesRequest): The incoming RPC request.
 +            context (grpc.ServicerContext): Context for the RPC call.
 +        """
 +        self.__logger.debug("GetCapabilities request from [%s]", context.peer())
++
          try:
              instance = self._get_instance(request.instance_name)
++
              return instance.get_capabilities()
          except InvalidArgumentError as e:

buildgrid/server/cas/instance.py

@@ -34,10 +34,25 @@ class ContentAddressableStorageInstance:
      def __init__(self, storage):
          self.__logger = logging.getLogger(__name__)
 -        self._storage = storage
 +        self._instance_name = None
++
 +        self.__storage = storage
++
 +    # --- Public API ---
++
 +    @property
 +    def instance_name(self):
 +        return self._instance_name
      def register_instance_with_server(self, instance_name, server):
 -        server.add_cas_instance(self, instance_name)
 +        """Names and registers the CAS instance with a given server."""
 +        if self._instance_name is None:
 +            server.add_cas_instance(self, instance_name)
++
 +            self._instance_name = instance_name
++
 +        else:
 +            raise AssertionError("Instance already registered")
      def hash_type(self):
          return get_hash_type()
@@ -51,12 +66,12 @@ class ContentAddressableStorageInstance:
          return re_pb2.CacheCapabilities().DISALLOWED
      def find_missing_blobs(self, blob_digests):
 -        storage = self._storage
 +        storage = self.__storage
          return re_pb2.FindMissingBlobsResponse(
              missing_blob_digests=storage.missing_blobs(blob_digests))
      def batch_update_blobs(self, requests):
 -        storage = self._storage
 +        storage = self.__storage
          store = []
          for request_proto in requests:
              store.append((request_proto.digest, request_proto.data))
@@ -72,7 +87,7 @@ class ContentAddressableStorageInstance:
          return response
      def batch_read_blobs(self, digests):
 -        storage = self._storage
 +        storage = self.__storage
          response = re_pb2.BatchReadBlobsResponse()
@@ -101,7 +116,7 @@ class ContentAddressableStorageInstance:
          return response
      def get_tree(self, request):
 -        storage = self._storage
 +        storage = self.__storage
          response = re_pb2.GetTreeResponse()
          page_size = request.page_size
@@ -143,10 +158,25 @@ class ByteStreamInstance:
      def __init__(self, storage):
          self.__logger = logging.getLogger(__name__)
 -        self._storage = storage
 +        self._instance_name = None
++
 +        self.__storage = storage
++
 +    # --- Public API ---
++
 +    @property
 +    def instance_name(self):
 +        return self._instance_name
      def register_instance_with_server(self, instance_name, server):
 -        server.add_bytestream_instance(self, instance_name)
 +        """Names and registers the byte-stream instance with a given server."""
 +        if self._instance_name is None:
 +            server.add_bytestream_instance(self, instance_name)
++
 +            self._instance_name = instance_name
++
 +        else:
 +            raise AssertionError("Instance already registered")
      def read(self, digest_hash, digest_size, read_offset, read_limit):
          if len(digest_hash) != HASH_LENGTH or not digest_size.isdigit():
@@ -169,7 +199,7 @@ class ByteStreamInstance:
              raise InvalidArgumentError("Negative read_limit is invalid")
          # Read the blob from storage and send its contents to the client.
 -        result = self._storage.get_blob(digest)
 +        result = self.__storage.get_blob(digest)
          if result is None:
              raise NotFoundError("Blob not found")
@@ -191,7 +221,7 @@ class ByteStreamInstance:
          digest = re_pb2.Digest(hash=digest_hash, size_bytes=int(digest_size))
 -        write_session = self._storage.begin_write(digest)
 +        write_session = self.__storage.begin_write(digest)
          # Start the write session and write the first request's data.
          write_session.write(first_block)
@@ -213,6 +243,6 @@ class ByteStreamInstance:
          elif computed_hash.hexdigest() != digest.hash:
              raise InvalidArgumentError("Data does not match hash")
 -        self._storage.commit_write(digest, write_session)
 +        self.__storage.commit_write(digest, write_session)
          return bytestream_pb2.WriteResponse(committed_size=bytes_written)

buildgrid/server/controller.py

@@ -36,19 +36,19 @@ from .operations.instance import OperationsInstance
  class ExecutionController:
 -    def __init__(self, action_cache=None, storage=None):
 +    def __init__(self, storage=None, action_cache=None, action_browser_url=None):
          self.__logger = logging.getLogger(__name__)
 -        scheduler = Scheduler(action_cache)
 +        scheduler = Scheduler(action_cache, action_browser_url=action_browser_url)
          self._execution_instance = ExecutionInstance(scheduler, storage)
          self._bots_interface = BotsInterface(scheduler)
          self._operations_instance = OperationsInstance(scheduler)
      def register_instance_with_server(self, instance_name, server):
 -        server.add_execution_instance(self._execution_instance, instance_name)
 -        server.add_bots_interface(self._bots_interface, instance_name)
 -        server.add_operations_instance(self._operations_instance, instance_name)
 +        self._execution_instance.register_instance_with_server(instance_name, server)
 +        self._bots_interface.register_instance_with_server(instance_name, server)
 +        self._operations_instance.register_instance_with_server(instance_name, server)
      def stream_operation_updates(self, message_queue, operation_name):
          operation = message_queue.get()

buildgrid/server/execution/instance.py

@@ -31,15 +31,32 @@ class ExecutionInstance:
      def __init__(self, scheduler, storage):
          self.__logger = logging.getLogger(__name__)
 -        self._storage = storage
          self._scheduler = scheduler
 +        self._instance_name = None
++
 +        self.__storage = storage
++
 +    # --- Public API ---
++
 +    @property
 +    def instance_name(self):
 +        return self._instance_name
      @property
      def scheduler(self):
          return self._scheduler
      def register_instance_with_server(self, instance_name, server):
 -        server.add_execution_instance(self, instance_name)
 +        """Names and registers the execution instance with a given server."""
 +        if self._instance_name is None:
 +            server.add_execution_instance(self, instance_name)
++
 +            self._instance_name = instance_name
 +            if self._scheduler is not None:
 +                self._scheduler.set_instance_name(instance_name)
++
 +        else:
 +            raise AssertionError("Instance already registered")
      def hash_type(self):
          return get_hash_type()
@@ -49,11 +66,12 @@ class ExecutionInstance:
          Queues an action and creates an Operation instance to be associated with
          this action.
          """
 -        action = self._storage.get_message(action_digest, Action)
 +        action = self.__storage.get_message(action_digest, Action)
++
          if not action:
              raise FailedPreconditionError("Could not get action from storage.")
 -        command = self._storage.get_message(action.command_digest, Command)
 +        command = self.__storage.get_message(action.command_digest, Command)
          if not command:
              raise FailedPreconditionError("Could not get command from storage.")

buildgrid/server/job.py

@@ -37,10 +37,8 @@ class Job:
          self._action = remote_execution_pb2.Action()
          self._lease = None
 -        self.__execute_response = None
 +        self.__execute_response = remote_execution_pb2.ExecuteResponse()
          self.__operation_metadata = remote_execution_pb2.ExecuteOperationMetadata()
 -        self.__operations_by_name = {}  # Name to Operation 1:1 mapping
 -        self.__operations_by_peer = {}  # Peer to Operation 1:1 mapping
          self.__queued_timestamp = timestamp_pb2.Timestamp()
          self.__queued_time_duration = duration_pb2.Duration()
@@ -48,6 +46,8 @@ class Job:
          self.__worker_completed_timestamp = timestamp_pb2.Timestamp()
          self.__operations_message_queues = {}
 +        self.__operations_by_name = {}  # Name to Operation 1:1 mapping
 +        self.__operations_by_peer = {}  # Peer to Operation 1:1 mapping
          self.__operations_cancelled = set()
          self.__lease_cancelled = False
          self.__job_cancelled = False
@@ -146,6 +146,11 @@ class Job:
          else:
              return False
 +    def set_action_url(self, url):
 +        """Generates a CAS browser URL for the job's action."""
 +        if url.for_message('action', self.__operation_metadata.action_digest):
 +            self.__execute_response.message = url.generate()
++
      def set_cached_result(self, action_result):
          """Allows specifying an action result form the action cache for the job.
@@ -155,7 +160,6 @@ class Job:
          Args:
              action_result (ActionResult): The result from cache.
          """
 -        self.__execute_response = remote_execution_pb2.ExecuteResponse()
          self.__execute_response.result.CopyFrom(action_result)
          self.__execute_response.cached_result = True
@@ -445,7 +449,6 @@ class Job:
              action_metadata.worker_start_timestamp.CopyFrom(self.__worker_start_timestamp)
              action_metadata.worker_completed_timestamp.CopyFrom(self.__worker_completed_timestamp)
 -            self.__execute_response = remote_execution_pb2.ExecuteResponse()
              self.__execute_response.result.CopyFrom(action_result)
              self.__execute_response.cached_result = False
              self.__execute_response.status.CopyFrom(status)

buildgrid/server/operations/instance.py

@@ -31,13 +31,27 @@ class OperationsInstance:
          self.__logger = logging.getLogger(__name__)
          self._scheduler = scheduler
 +        self._instance_name = None
++
 +    # --- Public API ---
++
 +    @property
 +    def instance_name(self):
 +        return self._instance_name
      @property
      def scheduler(self):
          return self._scheduler
      def register_instance_with_server(self, instance_name, server):
 -        server.add_operations_instance(self, instance_name)
 +        """Names and registers the operations instance with a given server."""
 +        if self._instance_name is None:
 +            server.add_operations_instance(self, instance_name)
++
 +            self._instance_name = instance_name
++
 +        else:
 +            raise AssertionError("Instance already registered")
      def get_operation(self, job_name):
          try:

buildgrid/server/referencestorage/storage.py

@@ -41,13 +41,29 @@ class ReferenceCache:
          """
          self.__logger = logging.getLogger(__name__)
 +        self._instance_name = None
++
 +        self.__storage = storage
++
          self._allow_updates = allow_updates
 -        self._storage = storage
          self._max_cached_refs = max_cached_refs
          self._digest_map = collections.OrderedDict()
 +    # --- Public API ---
++
 +    @property
 +    def instance_name(self):
 +        return self._instance_name
++
      def register_instance_with_server(self, instance_name, server):
 -        server.add_reference_storage_instance(self, instance_name)
 +        """Names and registers the refs instance with a given server."""
 +        if self._instance_name is None:
 +            server.add_reference_storage_instance(self, instance_name)
++
 +            self._instance_name = instance_name
++
 +        else:
 +            raise AssertionError("Instance already registered")
      @property
      def allow_updates(self):
@@ -64,7 +80,8 @@ class ReferenceCache:
              NotFoundError.
          """
          if key in self._digest_map:
 -            reference_result = self._storage.get_message(self._digest_map[key], remote_execution_pb2.Digest)
 +            reference_result = self.__storage.get_message(self._digest_map[key],
 +                                                          remote_execution_pb2.Digest)
              if reference_result is not None:
                  return reference_result
@@ -84,7 +101,8 @@ class ReferenceCache:
              NotFoundError.
          """
          if key in self._digest_map:
 -            reference_result = self._storage.get_message(self._digest_map[key], remote_execution_pb2.ActionResult)
 +            reference_result = self.__storage.get_message(self._digest_map[key],
 +                                                          remote_execution_pb2.ActionResult)
              if reference_result is not None:
                  if self._action_result_blobs_still_exist(reference_result):
@@ -115,9 +133,11 @@ class ReferenceCache:
          while len(self._digest_map) >= self._max_cached_refs:
              self._digest_map.popitem(last=False)
 -        result_digest = self._storage.put_message(result)
 +        result_digest = self.__storage.put_message(result)
          self._digest_map[key] = result_digest
 +    # --- Private API ---
++
      def _action_result_blobs_still_exist(self, action_result):
          """Checks CAS for ActionResult output blobs existance.
@@ -135,8 +155,8 @@ class ReferenceCache:
          for output_directory in action_result.output_directories:
              blobs_needed.append(output_directory.tree_digest)
 -            tree = self._storage.get_message(output_directory.tree_digest,
 -                                             remote_execution_pb2.Tree)
 +            tree = self.__storage.get_message(output_directory.tree_digest,
 +                                              remote_execution_pb2.Tree)
              if tree is None:
                  return False
@@ -153,5 +173,5 @@ class ReferenceCache:
          if action_result.stderr_digest.hash and not action_result.stderr_raw:
              blobs_needed.append(action_result.stderr_digest)
 -        missing = self._storage.missing_blobs(blobs_needed)
 +        missing = self.__storage.missing_blobs(blobs_needed)
          return len(missing) == 0

buildgrid/server/scheduler.py

@@ -22,19 +22,23 @@ Schedules jobs.
  import bisect
  from datetime import timedelta
  import logging
 +from threading import Lock
  from buildgrid._enums import LeaseState, OperationStage
  from buildgrid._exceptions import NotFoundError
  from buildgrid.server.job import Job
 +from buildgrid.utils import BrowserURL
  class Scheduler:
      MAX_N_TRIES = 5
 -    def __init__(self, action_cache=None, monitor=False):
 +    def __init__(self, action_cache=None, action_browser_url=False, monitor=False):
          self.__logger = logging.getLogger(__name__)
 +        self._instance_name = None
++
          self.__build_metadata_queues = None
          self.__operations_by_stage = None
@@ -43,20 +47,29 @@ class Scheduler:
          self.__retries_count = 0
          self._action_cache = action_cache
 +        self._action_browser_url = action_browser_url
          self.__jobs_by_action = {}  # Action to Job 1:1 mapping
          self.__jobs_by_operation = {}  # Operation to Job 1:1 mapping
          self.__jobs_by_name = {}  # Name to Job 1:1 mapping
          self.__queue = []
 +        self.__queue_lock = Lock()
 -        self._is_instrumented = monitor
+-
 -        if self._is_instrumented:
 +        self._is_instrumented = False
 +        if monitor:
              self.activate_monitoring()
      # --- Public API ---
 +    @property
 +    def instance_name(self):
 +        return self._instance_name
++
 +    def set_instance_name(self, instance_name):
 +        if not self._instance_name:
 +            self._instance_name = instance_name
++
      def list_current_jobs(self):
          """Returns a list of the :class:`Job` names currently managed."""
          return self.__jobs_by_name.keys()
@@ -186,6 +199,10 @@ class Scheduler:
                    platform_requirements=platform_requirements,
                    priority=priority)
 +        if self._action_browser_url:
 +            job.set_action_url(
 +                BrowserURL(self._action_browser_url, self._instance_name))
++
          self.__logger.debug("Job created for action [%s]: [%s]",
                              action_digest.hash[:8], job.name)
@@ -281,27 +298,26 @@ class Scheduler:
                  worker properties, configuration and state at the time of the
                  request.
          """
 -        if not self.__queue:
 -            return []
+-
 -        # Looking for the first job that could be assigned to the worker...
 -        for job_index, job in enumerate(self.__queue):
 -            if self._worker_is_capable(worker_capabilities, job):
 -                self.__logger.info("Job scheduled to run: [%s]", job.name)
 +        # TODO: Replace with a more efficient way of doing this.
 +        with self.__queue_lock:
 +            # Looking for the first job that could be assigned to the worker...
 +            for job_index, job in enumerate(self.__queue):
 +                if self._worker_is_capable(worker_capabilities, job):
 +                    self.__logger.info("Job scheduled to run: [%s]", job.name)
 -                lease = job.lease
 +                    lease = job.lease
 -                if not lease:
 -                    # For now, one lease at a time:
 -                    lease = job.create_lease()
 +                    if not lease:
 +                        # For now, one lease at a time:
 +                        lease = job.create_lease()
 -                if lease:
 -                    del self.__queue[job_index]
 -                    return [lease]
 +                    if lease:
 +                        del self.__queue[job_index]
 +                        return [lease]
 -                return None
 +                    return []
 -        return None
 +            return []
      def update_job_lease_state(self, job_name, lease):
          """Requests a state transition for a job's current :class:Lease.
@@ -535,11 +551,12 @@ class Scheduler:
          """Schedules or reschedules a job."""
          job = self.__jobs_by_name[job_name]
 -        if job.operation_stage == OperationStage.QUEUED:
 -            self.__queue.sort()
 +        with self.__queue_lock:
 +            if job.operation_stage == OperationStage.QUEUED:
 +                self.__queue.sort()
 -        else:
 -            bisect.insort(self.__queue, job)
 +            else:
 +                bisect.insort(self.__queue, job)
          self.__logger.info("Job queued: [%s]", job.name)
@@ -548,7 +565,8 @@ class Scheduler:
          job = self.__jobs_by_name[job_name]
          if job.operation_stage == OperationStage.QUEUED:
 -            self.__queue.remove(job)
 +            with self.__queue_lock:
 +                self.__queue.remove(job)
          del self.__jobs_by_action[job.action_digest.hash]
          del self.__jobs_by_name[job.name]

buildgrid/settings.py

@@ -16,10 +16,16 @@
  import hashlib
 +# Latest REAPI version supported:
 +HIGH_REAPI_VERSION = '2.0.0'
++
 +# Earliest non-deprecated REAPI version supported:
 +LOW_REAPI_VERSION = '2.0.0'
++
  # Hash function used for computing digests:
  HASH = hashlib.sha256
 -# Lenght in bytes of a hash string returned by HASH:
 +# Length in bytes of a hash string returned by HASH:
  HASH_LENGTH = HASH().digest_size * 2
  # Period, in seconds, for the monitoring cycle:
@@ -31,7 +37,20 @@ MAX_REQUEST_SIZE = 2 * 1024 * 1024
  # Maximum number of elements per gRPC request:
  MAX_REQUEST_COUNT = 500
 +# Value that establishes an upper bound on the size of a file that can
 +# be queued into a batch request. Expressed as a percentage of the
 +# batch size limit:
 +BATCH_REQUEST_SIZE_THRESHOLD = 0.25
++
  # String format for log records:
  LOG_RECORD_FORMAT = '%(asctime)s:[%(name)36.36s][%(levelname)5.5s]: %(message)s'
  # The different log record attributes are documented here:
  # https://docs.python.org/3/library/logging.html#logrecord-attributes
++
 +# URL scheme for the CAS content browser:
 +BROWSER_URL_FORMAT = '%(type)s/%(instance)s/%(hash)s/%(sizebytes)s/'
 +# The string markers that are substituted are:
 +#  instance   - CAS instance's name.
 +#  type       - Type of CAS object, eg. 'action_result', 'command'...
 +#  hash       - Object's digest hash.
 +#  sizebytes  - Object's digest size in bytes.

buildgrid/utils.py

@@ -13,14 +13,61 @@
  # limitations under the License.
 +from urllib.parse import urljoin
  from operator import attrgetter
  import os
  import socket
 -from buildgrid.settings import HASH, HASH_LENGTH
 +from buildgrid.settings import HASH, HASH_LENGTH, BROWSER_URL_FORMAT
  from buildgrid._protos.build.bazel.remote.execution.v2 import remote_execution_pb2
 +class BrowserURL:
++
 +    __url_markers = (
 +        '%(instance)s',
 +        '%(type)s',
 +        '%(hash)s',
 +        '%(sizebytes)s',
 +    )
++
 +    def __init__(self, base_url, instance_name=None):
 +        """Begins browser URL helper initialization."""
 +        self.__base_url = base_url
 +        self.__initialized = False
 +        self.__url_spec = {
 +            '%(instance)s': instance_name or '',
 +        }
++
 +    def for_message(self, message_type, message_digest):
 +        """Completes browser URL initialization for a protobuf message."""
 +        if self.__initialized:
 +            return False
++
 +        self.__url_spec['%(type)s'] = message_type
 +        self.__url_spec['%(hash)s'] = message_digest.hash
 +        self.__url_spec['%(sizebytes)s'] = str(message_digest.size_bytes)
++
 +        self.__initialized = True
 +        return True
++
 +    def generate(self):
 +        """Generates a browser URL string."""
 +        if not self.__base_url or not self.__initialized:
 +            return None
++
 +        url_tail = BROWSER_URL_FORMAT
++
 +        for url_marker in self.__url_markers:
 +            if url_marker not in self.__url_spec:
 +                return None
 +            if url_marker not in url_tail:
 +                continue
 +            url_tail = url_tail.replace(url_marker, self.__url_spec[url_marker])
++
 +        return urljoin(self.__base_url, url_tail)
++
++
  def get_hostname():
      """Returns the hostname of the machine executing that function.

tests/integration/execution_service.py

@@ -66,10 +66,10 @@ def controller(request):
      if request.param == "action-cache":
          cache = ActionCache(storage, 50)
 -        yield ExecutionController(cache, storage)
 +        yield ExecutionController(storage=storage, action_cache=cache)
      else:
 -        yield ExecutionController(None, storage)
 +        yield ExecutionController(storage=storage)
  # Instance to test

tests/integration/operations_service.py

@@ -71,7 +71,7 @@ def controller():
      write_session.write(action.SerializeToString())
      storage.commit_write(action_digest, write_session)
 -    yield ExecutionController(None, storage)
 +    yield ExecutionController(storage=storage)
  # Instance to test

tests/test_utils.py

 +# Copyright (C) 2018 Bloomberg LP
 +#
 +# Licensed under the Apache License, Version 2.0 (the "License");
 +# you may not use this file except in compliance with the License.
 +# You may obtain a copy of the License at
 +#
 +#  <http://www.apache.org/licenses/LICENSE-2.0>
 +#
 +# Unless required by applicable law or agreed to in writing, software
 +# distributed under the License is distributed on an "AS IS" BASIS,
 +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 +# See the License for the specific language governing permissions and
 +# limitations under the License.
++
++
 +from urllib.parse import urlparse
++
 +import pytest
++
 +from buildgrid._protos.build.bazel.remote.execution.v2 import remote_execution_pb2
 +from buildgrid.utils import BrowserURL
 +from buildgrid.utils import get_hash_type
 +from buildgrid.utils import create_digest, parse_digest
++
++
 +BLOBS = (b'', b'non-empty-blob',)
 +BLOB_HASHES = (
 +    'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855',
 +    '89070dfb3175a2c75835d70147b52bd97afd8228819566d84eecd2d20e9b19fc',)
 +BLOB_SIZES = (0, 14,)
 +BLOB_DATA = zip(BLOBS, BLOB_HASHES, BLOB_SIZES)
++
 +STRINGS = (
 +    'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855/0',
 +    '89070dfb3175a2c75835d70147b52bd97afd8228819566d84eecd2d20e9b19fc/14',
 +    'e1ca41574914ba00e8ed5c8fc78ec8efdfd48941c7e48ad74dad8ada7f2066d/12', )
 +BLOB_HASHES = (
 +    'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855',
 +    '89070dfb3175a2c75835d70147b52bd97afd8228819566d84eecd2d20e9b19fc',
 +    None, )
 +BLOB_SIZES = (0, 14, None,)
 +STRING_VALIDITIES = (True, True, False,)
 +STRING_DATA = zip(STRINGS, BLOB_HASHES, BLOB_SIZES, STRING_VALIDITIES)
++
 +BASE_URL = 'http://localhost:8080'
 +INSTANCES = (None, '', 'instance',)
 +URL_HASHES = (
 +    'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855',
 +    '89070dfb3175a2c75835d70147b52bd97afd8228819566d84eecd2d20e9b19fc',)
 +URL_SIZES = (0, 14,)
 +URL_DATA = zip(URL_HASHES, URL_SIZES)
++
++
 +@pytest.mark.parametrize('blob,digest_hash,digest_size', BLOB_DATA)
 +def test_create_digest(blob, digest_hash, digest_size):
 +    # Generate a Digest message from given blob:
 +    blob_digest = create_digest(blob)
++
 +    assert get_hash_type() == remote_execution_pb2.SHA256
++
 +    assert hasattr(blob_digest, 'DESCRIPTOR')
 +    assert blob_digest.DESCRIPTOR == remote_execution_pb2.Digest.DESCRIPTOR
 +    assert blob_digest.hash == digest_hash
 +    assert blob_digest.size_bytes == digest_size
++
++
 +@pytest.mark.parametrize('string,digest_hash,digest_size,validity', STRING_DATA)
 +def test_parse_digest(string, digest_hash, digest_size, validity):
 +    # Generate a Digest message from given string:
 +    string_digest = parse_digest(string)
++
 +    assert get_hash_type() == remote_execution_pb2.SHA256
++
 +    if validity:
 +        assert hasattr(string_digest, 'DESCRIPTOR')
 +        assert string_digest.DESCRIPTOR == remote_execution_pb2.Digest.DESCRIPTOR
 +        assert string_digest.hash == digest_hash
 +        assert string_digest.size_bytes == digest_size
++
 +    else:
 +        assert string_digest is None
++
++
 +@pytest.mark.parametrize('instance', INSTANCES)
 +@pytest.mark.parametrize('digest_hash,digest_size', URL_DATA)
 +def test_browser_url_initialization(instance, digest_hash, digest_size):
 +    # Initialize and generate a browser compatible URL:
 +    browser_url = BrowserURL(BASE_URL, instance)
 +    browser_digest = remote_execution_pb2.Digest(hash=digest_hash,
 +                                                 size_bytes=digest_size)
++
 +    assert browser_url.generate() is None
 +    assert browser_url.for_message('type', browser_digest)
 +    assert not browser_url.for_message(None, None)
++
 +    url = browser_url.generate()
++
 +    assert url is not None
++
 +    parsed_url = urlparse(url)
++
 +    if instance:
 +        assert parsed_url.path.find(instance)
 +    assert parsed_url.path.find('type') > 0
 +    assert parsed_url.path.find(digest_hash) > 0
 +    assert parsed_url.path.find(str(digest_size)) > 0

[Notes] [Git][BuildGrid/buildgrid][santigl/100-server-capabilities] 14 commits: Udapte .proto files to their latest version

Santiago Gil pushed to branch santigl/100-server-capabilities at BuildGrid / buildgrid

Commits:

23 changed files:

Changes: