[Notes] [Git][BuildGrid/buildgrid][cartr/buildbox-e2e] 35 commits: Downl

Jeremiah Bonney pushed to branch cartr/buildbox-e2e at BuildGrid / buildgrid

Commits:

ebc39792

by Santiago Gil at 2019-01-31T17:06:34Z

Downloader._write_directory(): request dirs in batches

Attempt to fetch directories in a given level using a single batch
request. If not possible to do so, fall back to using Read().

3fd9add5

by Jeremiah Bonney at 2019-02-07T17:01:49Z

Add an option to ActionCache to only store successful actions

Addresses Issue #151

5c10c0ad

by Jeremiah Bonney at 2019-02-07T17:01:52Z

Add test for new cache_failed_actions functionality

Addresses Issue #151

af8a5816

by Martin Blanchard at 2019-02-07T17:33:44Z

cli.py: Better handle errors during command invocations

https://gitlab.com/BuildGrid/buildgrid/issues/156

fb812017

by Martin Blanchard at 2019-02-07T17:40:11Z

actioncache/storage.py: Log cache update events

https://gitlab.com/BuildGrid/buildgrid/issues/156

ee5542ab

by Martin Blanchard at 2019-02-07T17:40:22Z

client/actioncache.py: Add simple ActionCache client

https://gitlab.com/BuildGrid/buildgrid/issues/156

3ad1d461

by Martin Blanchard at 2019-02-07T17:40:22Z

utils.py: Introduce a Digest string parser

https://gitlab.com/BuildGrid/buildgrid/issues/156

710da565

by Martin Blanchard at 2019-02-07T17:40:22Z

cmd_actioncache.py: New bgd-action-cache command

https://gitlab.com/BuildGrid/buildgrid/issues/156

b7355f32

by Martin Blanchard at 2019-02-07T17:40:22Z

docs: Include bgd-action-cache in CLI reference

https://gitlab.com/BuildGrid/buildgrid/issues/156

9717831a

by Santiago Gil at 2019-02-13T10:56:10Z

cmd_bot and cmd_execute: allow specifying list of Platform attributes

(To address #104)

1d5f956a

by Santiago Gil at 2019-02-13T10:56:10Z

Scheduler: match OS and ISA between job and worker

With these changes the `platform` attributes of the Commands and
workers start to be considered by the scheduler.

When a given worker requests a Job, the scheduler will now try and assign it
the first job in the queue that can be executed in that worker.

As a side effect, submitting an Action with no Command now will result
in a FailedPreconditionError exception.

(This will allow to address #104)

c44ec8a6

by Martin Blanchard at 2019-02-13T11:44:11Z

Udapte .proto files to their latest version

Noteworthy changes include:
* From remote_execution.proto:
  - New message field in ExecuteResponse [1].
  - Output file and dir. parent folders clarifications [2].
  - Better doc. for do_not_cache/skip_cache_lookup [3].
  - Documention updates.
* From semver.proto:
  - Documentation updates.

[1] https://github.com/bazelbuild/remote-apis/pull/36
[2] https://github.com/bazelbuild/remote-apis/pull/42
[3] https://github.com/bazelbuild/remote-apis/pull/55

1c60ccc1

by Martin Blanchard at 2019-02-14T13:49:54Z

Make server instances aware of their name

https://gitlab.com/BuildGrid/buildgrid/issues/157

56f129ed

by Martin Blanchard at 2019-02-14T13:49:57Z

controller.py: Fix server instances registration

Delegate server registration to actual instance code instead of trying
to get it right from the controller class.

https://gitlab.com/BuildGrid/buildgrid/issues/157

f07c8895

by Martin Blanchard at 2019-02-14T13:50:03Z

scheduler.py: Hold parent's instance name

https://gitlab.com/BuildGrid/buildgrid/issues/157

4e96eb89

by Martin Blanchard at 2019-02-14T13:50:06Z

utils.py: Introduce a CAS browser URL helper

https://gitlab.com/BuildGrid/buildgrid/issues/157

3720db50

by Martin Blanchard at 2019-02-14T13:50:10Z

Introduce an action browser URL config. key

https://gitlab.com/BuildGrid/buildgrid/issues/157

f768c632

by Martin Blanchard at 2019-02-14T13:50:14Z

Generate per-job action browser URLs

https://gitlab.com/BuildGrid/buildgrid/issues/157

d0a8c0bc

by Martin Blanchard at 2019-02-14T13:50:17Z

tests: Add simple tests utils and helpers

https://gitlab.com/BuildGrid/buildgrid/issues/157

78efe8c6

by Santiago Gil at 2019-02-18T16:32:52Z

Scheduler: add mutex to job queue

This solves race conditions in the scheduler (introduced by !151, which
seeked to address #104).

ed7ab756

by Martin Blanchard at 2019-02-19T16:23:03Z

capabilities/service.py: Log debug message on request

https://gitlab.com/BuildGrid/buildgrid/issues/161

95f83758

by Martin Blanchard at 2019-02-19T16:23:19Z

settings.py: Add supported versions for REAPI

https://gitlab.com/BuildGrid/buildgrid/issues/161

b9f59bf3

by Martin Blanchard at 2019-02-19T16:25:13Z

capabilities/instance.py: Advertise supported REAPI version

https://gitlab.com/BuildGrid/buildgrid/issues/161

4c971b76

by Santiago Gil at 2019-02-20T10:46:59Z

Scheduler: fix monitoring variables initialization

Calling `Scheduler.activate_monitoring()` from the `Scheduler`
constructor didn't have any effect because `self.activate_monitoring()`
always exited early.

This change fixes the boolean check before that function call so that
the monitoring structures get initialized properly.

4796f913

by Santiago Gil at 2019-02-20T15:24:09Z

CAS.{Up, Down}loader(): if provided, respect `CacheCapabilities`

Try fetching the `CacheCapabilities.max_batch_total_size_bytes` value
from the server and, if the server provides a limit, respect it for
batch transactions.

Also, make the threshold value for determining when to queue a file
for fetching in a batch request a configurable % of the maximum
batch request size.

(Addresses #100)

7d83fbb1