[Notes] [Git][BuildGrid/buildgrid][mablanch/75-requests-multiplexing] 123 commits: setup.py: Pin moto dependency and require < 1.3.7

[Martin Blanchard <gitlab mg gitlab com>]

Title: GitLab

Martin Blanchard pushed to branch mablanch/75-requests-multiplexing at BuildGrid / buildgrid

Commits:

• 589ffa40
  by Martin Blanchard at 2018-11-06T15:12:47Z

  setup.py: Pin moto dependency and require < 1.3.7
  
  https://gitlab.com/BuildGrid/buildgrid/issues/65
  

• 07963525
  by Raoul Hidalgo Charman at 2018-11-08T13:57:26Z

  Add logging to service gRPC requests
  

• aac2f938
  by Raoul Hidalgo Charman at 2018-11-08T13:57:26Z

  cli.py: Allow setting log verbosity level
  

• 76f2e31f
  by Martin Blanchard at 2018-11-08T13:57:26Z

  cli.py: Reset root logger at startup and use default format
  

• dcd82219
  by Martin Blanchard at 2018-11-08T14:50:25Z

  Mark class logger instances as private
  

• dd572340
  by Martin Blanchard at 2018-11-08T16:54:57Z

  Harmonise gRPC servicers debug messages for requests
  

• 6ca668f8
  by Martin Blanchard at 2018-11-12T09:12:58Z

  client/cas.py: Preserve file executability on download
  
  https://gitlab.com/BuildGrid/buildgrid/issues/134
  

• 3512e0bb
  by Martin Blanchard at 2018-11-12T09:13:08Z

  tests/cas/test_client.py: Add a test file for executability
  
  https://gitlab.com/BuildGrid/buildgrid/issues/134
  

• 886e3ff4
  by Laurence Urhegyi at 2018-11-22T18:18:58Z

  Update CONTRIBUTING guide, add COMMITTERS list
  
  Removes some references to a 'meritocratic' project and the Subversion
  policy as they were thought to be un-needed.
  

• 131c6d87
  by Finn at 2018-11-23T09:01:50Z

  buildgrid/_app/commands/cmd_execute.py: Replace logger with execute.
  

• 2360d613
  by Finn at 2018-11-23T09:01:50Z

  buildgrid/_app/commands/cmd_cas.py: Replaced logger with echo.
  

• 15e7a095
  by Finn at 2018-11-23T09:01:50Z

  buildgrid/_app/commands/cmd_operation.py: Replacing logger with echo.
  

• b21c1258
  by Finn at 2018-11-23T09:01:50Z

  buildgrid/_app/commands/cmd_server.py: Replace logger with echo.
  

• 7e184bf9
  by Finn at 2018-11-23T09:01:50Z

  buildgrid/_app/commands/cmd_bot.py: Replaced logger with echo.
  

• 38ed83ba
  by Finn at 2018-11-23T09:01:50Z

  buildgrid_app/bots/: Removed reliance on context for logger.
  

• d0654fea
  by Finn at 2018-11-23T10:47:23Z

  Explictly state the OK code.
  

• 5d88a781
  by Finn at 2018-11-23T10:47:23Z

  Added CancelledError.
  
  This can be used to inform callers that the operation is cancelled.
  

• 152672f0
  by Finn at 2018-11-23T10:47:23Z

  buildgrid/server/execution/service.py: Catch CancelledError.
  

• c21f8116
  by Finn at 2018-11-23T10:47:23Z

  buildgrid/server/operations/service.py: Removed NotImplemented catch.
  

• fe7244f5
  by Finn at 2018-11-23T10:47:23Z

  Implement cancelletion methods in job class.
  

• a938d5f2
  by Finn at 2018-11-23T10:47:23Z

  Added cancellation method to scheduler.
  

• db3e57ac
  by Finn at 2018-11-23T10:47:23Z

  Added cancellation to operations instance.
  

• 555d486f
  by Finn at 2018-11-23T10:47:23Z

  Message queue now queues the job.
  
  Check for operation status when streaming operations.
  Allows client to catch a cancelled job.
  

• 3c51eebc
  by Finn at 2018-11-23T10:54:09Z

  Added cancel command to app.
  

• 5877fe6e
  by Finn at 2018-11-23T10:54:14Z

  tests/integration/operations_service.py: Ammended for cancellation.
  

• c917480a
  by Finn at 2018-11-23T11:18:49Z

  Scheduler can now cancel leases.
  

• fd0c1534
  by Finn at 2018-11-23T11:18:49Z

  BotsInterface now handles cancelled leases.
  
  Removed state machine. Will help with #66.
  

• 1e58b538
  by Finn at 2018-11-23T11:18:49Z

  Added HardwareInterface class.
  
  This will describe and configure hardware.
  

• b7d62408
  by Finn at 2018-11-23T11:18:49Z

  Add Device class.
  

• 17db2b8c
  by Finn at 2018-11-23T11:18:49Z

  Add Worker class.
  

• 694c42ab
  by Finn at 2018-11-23T11:18:49Z

  Remove hardware descriptions from bot_session.
  
  Also move `bot_session` to simply `session`.
  

• 87411369
  by Finn at 2018-11-23T11:41:53Z

  Added Tenant class.
  
  This will contain the information about a running task
  and lease.
  

• b031d8de
  by Finn at 2018-11-23T11:55:31Z

  Added TenantManager class.
  
  This will manage the Tenant objects.
  

• 663f1fd8
  by Finn at 2018-11-23T11:55:31Z

  Modified BotSession to use new TenantManager class.
  

• e8972884
  by Finn at 2018-11-23T15:02:39Z

  buildgrid/bot/bot.py: Close tasks when KeyboardInterupt.
  

• d1cef783
  by Finn at 2018-11-23T15:02:44Z

  Added exception handling to BotInterface.
  
  Also Moved bot_interface.py to interface.py.
  

• b0a1c9c9
  by Finn at 2018-11-23T15:11:01Z

  Moved work function and context argument order.
  
  Also moved the initialisation of these variables to
  the `__init__` of `session.py`.
  

• 1dd8291a
  by Finn at 2018-11-23T15:11:01Z

  Subprocess functions now in separate utils file.
  

• 63e2feaf
  by Finn at 2018-11-23T15:11:01Z

  Updated imports for function `run_in_subprocess`
  

• 24f4117f
  by Finn at 2018-11-23T15:11:01Z

  tests/utils/bots_interface.py: Added new bots_interface mock.
  

• cfdb72d9
  by Finn at 2018-11-23T15:11:01Z

  tests/integration/bot_session.py: Updated to use new subprocess tests.
  
  Also adds cancellation tests.
  

• 6b779d3a
  by Finn at 2018-11-23T15:11:01Z

  tests/integration/bots_service.py: Removed redundant tests.
  
  Mostly due to removing the state machine in the BotsService
  

• f9261257
  by Finn at 2018-11-23T15:11:01Z

  tests/integration/operations_service.py: Removing redundant test.
  

• 96eef9a5
  by Finn at 2018-11-23T15:11:01Z

  Added unittests for hardware interface.
  

• 45908b7e
  by Martin Blanchard at 2018-11-26T10:17:22Z

  storage_abc.py: Fix unnecessary pass linting issues
  

• 144df7b0
  by Martin Blanchard at 2018-11-26T10:23:32Z

  tests: Fix literal comparison linting issues
  

• 6c540071
  by Martin Blanchard at 2018-11-26T10:30:08Z

  Add private .proto file for monitoring bus messages
  
  https://gitlab.com/BuildGrid/buildgrid/issues/135
  

• 96c6281e
  by Martin Blanchard at 2018-11-26T10:30:08Z

  _enums.py: Expose Python Enums for monitoring.proto
  
  https://gitlab.com/BuildGrid/buildgrid/issues/135
  

• bab83a4b
  by Martin Blanchard at 2018-11-26T10:30:08Z

  monitoring.py: New monitoring bus class
  
  https://gitlab.com/BuildGrid/buildgrid/issues/135
  

• 1f5d9210
  by Martin Blanchard at 2018-11-26T10:30:08Z

  Handle the server's main event loop internally
  
  This turns Server.start() into a blocking call!
  
  https://gitlab.com/BuildGrid/buildgrid/issues/135
  

• e102e2d3
  by Martin Blanchard at 2018-11-26T10:30:08Z

  server/instance.py: Run monitoring bus in main event loop
  
  https://gitlab.com/BuildGrid/buildgrid/issues/135
  

• 31533b69
  by Martin Blanchard at 2018-11-26T10:30:08Z

  server/instance.py: Handle SIGTERM gracefully
  
  https://gitlab.com/BuildGrid/buildgrid/issues/135
  

• cc1676ed
  by Martin Blanchard at 2018-11-26T10:30:08Z

  server/instance.py: Return number of bound port
  
  Useful for tests, when port number is not determined beforehand.
  
  https://gitlab.com/BuildGrid/buildgrid/issues/135
  

• 1a8dff34
  by Martin Blanchard at 2018-11-26T10:30:08Z

  cas/service.py: Better parse resource names
  
  https://gitlab.com/BuildGrid/buildgrid/issues/135
  

• 66119b01
  by Martin Blanchard at 2018-11-26T10:30:08Z

  tests/utils/server.py: New server helper
  
  https://gitlab.com/BuildGrid/buildgrid/issues/135
  

• 40a7a09a
  by Martin Blanchard at 2018-11-26T10:30:08Z

  test/server_instance.py: Adapt to new blocking API
  
  https://gitlab.com/BuildGrid/buildgrid/issues/135
  

• 859c0fa8
  by Finn at 2018-11-27T15:25:25Z

  buildgrid/utils.py: New `get_hash_type` function.
  
  Returns the hash type.
  

• e1091b04
  by Finn at 2018-11-27T15:25:25Z

  Execution instance can now return hash type.
  

• 1dc5d2d2
  by Finn at 2018-11-27T15:25:25Z

  Adding cache capabilities to CAS instance.
  
  Hash type, batch size and symlink strategy.
  

• 35c901bd
  by Finn at 2018-11-27T16:23:49Z

  Adding capabilities service.
  

• 5cb38e2a
  by Finn at 2018-11-27T16:23:52Z

  Adding capabilities instance of service.
  

• 32ad653d
  by Finn at 2018-11-27T16:23:52Z

  Adding client interface for capabilities service.
  

• e15a9c91
  by Finn at 2018-11-27T16:23:52Z

  Capabilities service can now be added to server.
  

• bd5587ea
  by Finn at 2018-11-27T16:23:52Z

  tests/utils/capabilities.py: Adding service for tests.
  

• d94fa258
  by Finn at 2018-11-27T16:23:52Z

  tests/integration/capabilities_service.py: Creating tests.
  

• 6f90a553
  by Finn at 2018-11-27T16:23:52Z

  buildgrid/_app/commands/cmd_capabilities.py: New command.
  

• db65c5ec
  by Raoul Hidalgo Charman at 2018-11-28T12:23:41Z

  test_storage.py: change import order to appease pylint
  
  Newer versions of pylint complain.
  

• db53ffbc
  by Martin Blanchard at 2018-11-29T08:59:48Z

  execution/service.py: Calculate client counts
  
  https://gitlab.com/BuildGrid/buildgrid/issues/132
  

• 5ecfb7f8
  by Martin Blanchard at 2018-11-29T08:59:48Z

  bots/service.py: Calculate bot counts
  
  https://gitlab.com/BuildGrid/buildgrid/issues/132
  

• df5b6a80
  by Martin Blanchard at 2018-11-29T08:59:48Z

  Expose scheduler from controller's service instances
  
  https://gitlab.com/BuildGrid/buildgrid/issues/132
  

• 5e608d6b
  by Martin Blanchard at 2018-11-29T08:59:48Z

  .pylintrc: Add Duration to list of ignored classes
  
  https://gitlab.com/BuildGrid/buildgrid/issues/132
  

• c167a1d0
  by Martin Blanchard at 2018-11-29T08:59:48Z

  server/job.py: Calculate job queue time and retry count
  
  https://gitlab.com/BuildGrid/buildgrid/issues/132
  

• 8fc6d17d
  by Martin Blanchard at 2018-11-29T08:59:48Z

  server/scheduler.py: Calculate job counts
  
  https://gitlab.com/BuildGrid/buildgrid/issues/132
  

• 397f385b
  by Martin Blanchard at 2018-11-29T08:59:48Z

  setup.py: Introduce janus dependency (Python >= 3.5.3)
  
  https://gitlab.com/BuildGrid/buildgrid/issues/132
  

• dbbcdb50
  by Martin Blanchard at 2018-11-29T08:59:48Z

  settings.py: Add a monitoring period setting
  
  https://gitlab.com/BuildGrid/buildgrid/issues/132
  

• 50f3f63b
  by Martin Blanchard at 2018-11-29T08:59:48Z

  server/instance.py: Run a monitoring task periodically
  
  https://gitlab.com/BuildGrid/buildgrid/issues/132
  

• 100e91b9
  by Arber Xhindoli at 2018-11-30T16:40:19Z

  Implementation of the getTree method.
  

• 49586d88
  by Martin Blanchard at 2018-11-30T17:19:40Z

  _monitoring.py: Ensure buffer gets flushed in non-async mode
  
  https://gitlab.com/BuildGrid/buildgrid/issues/23
  

• 5673b009
  by Martin Blanchard at 2018-11-30T17:20:19Z

  settings.py: Add a log record format setting
  
  https://gitlab.com/BuildGrid/buildgrid/issues/23
  

• b763ec5f
  by Martin Blanchard at 2018-11-30T17:20:19Z

  .gitlab-ci.yml: Drop verbose mode during tests
  
  https://gitlab.com/BuildGrid/buildgrid/issues/23
  

• 76459e0a
  by Martin Blanchard at 2018-11-30T17:20:19Z

  Move --verbose CLI option do relevant subcommands
  
  https://gitlab.com/BuildGrid/buildgrid/issues/23
  

• 94bf76a7
  by Martin Blanchard at 2018-11-30T17:20:19Z

  cli.py: Allow filtering log record on their domains
  
  This patch allows filtering the log records that get printed to stdout
  and stderr based on their domain name using the BGD_MESSAGE_DEBUG
  environment variable. A colon separated list of domains is expected:
  
    BGD_MESSAGE_DEBUG="buildgrid.server.cas:buildgrid.client.cas"
  
  https://gitlab.com/BuildGrid/buildgrid/issues/23
  

• 8f5d71c5
  by Martin Blanchard at 2018-11-30T17:20:19Z

  server/instance.py: Setup a logging task at startup
  
  https://gitlab.com/BuildGrid/buildgrid/issues/23
  

• 1dde5014
  by Martin Blanchard at 2018-12-04T14:34:08Z

  job.py: Expose its ActionResult origin
  
  https://gitlab.com/BuildGrid/buildgrid/issues/139
  

• b4ff86bb
  by Martin Blanchard at 2018-12-06T10:05:21Z

  scheduler.py: Allow registering for build metadata updates
  
  https://gitlab.com/BuildGrid/buildgrid/issues/139
  

• c5f39b31
  by Martin Blanchard at 2018-12-06T10:07:56Z

  server/instance.py: Register tasks for build metrics
  
  https://gitlab.com/BuildGrid/buildgrid/issues/139
  

• 14d8ae0e
  by Martin Blanchard at 2018-12-06T10:09:27Z

  server/instance.py: Stop using 'void' for empty instance in metrics
  
  https://gitlab.com/BuildGrid/buildgrid/issues/139
  

• 38eb6692
  by Raoul Hidalgo Charman at 2018-12-10T11:20:10Z

  bot: Add logic to reconnect
  
  Bot will call CreateBotSession or UpdateBotSession depending on whether it is
  still connected to the server. Added run method to bot session that does this
  logic.
  
  Fixes #126
  

• 6ec6bf57
  by Raoul Hidalgo Charman at 2018-12-10T11:20:10Z

  cmd_bot.py: Remove unnecessary KeyboardInterrupt handlers
  
  There's already one in bot session so don't need one in cmd_bot
  

• 58f0dfaf
  by Martin Blanchard at 2018-12-10T11:30:51Z

  setup.py: Introduce pyjwt dependency
  
  Hosted at: https://github.com/jpadilla/pyjwt
  
  The dependency is declared as part of a new 'auth' extra target so that
  it is not installed by default, but only if requested.
  
  https://gitlab.com/BuildGrid/buildgrid/issues/144
  

• ad6c5848
  by Martin Blanchard at 2018-12-10T11:30:51Z

  server/_authentication.py: New JWT based gRPC interceptor
  
  https://gitlab.com/BuildGrid/buildgrid/issues/144
  

• 0233420b
  by Martin Blanchard at 2018-12-10T11:30:51Z

  server/instance.py: Register authentication gRPC interceptor
  
  https://gitlab.com/BuildGrid/buildgrid/issues/144
  

• 5e8c5dbd
  by Martin Blanchard at 2018-12-10T11:30:51Z

  client/authentication.py: New client gRPC channel helpers
  
  https://gitlab.com/BuildGrid/buildgrid/issues/144
  

• dac3e8ea
  by Martin Blanchard at 2018-12-10T11:30:51Z

  cmd_bot.py: Port to new client channel helpers
  
  https://gitlab.com/BuildGrid/buildgrid/issues/144
  

• d9071db2
  by Martin Blanchard at 2018-12-10T11:30:51Z

  cmd_capabilities.py: Port to new client channel helpers
  
  https://gitlab.com/BuildGrid/buildgrid/issues/144
  

• 9cf39c9a
  by Martin Blanchard at 2018-12-10T11:30:51Z

  cmd_cas.py: Port to new client channel helpers
  
  https://gitlab.com/BuildGrid/buildgrid/issues/144
  

• d5fceafa
  by Martin Blanchard at 2018-12-10T11:30:51Z

  cmd_execute.py: Port to new client channel helpers
  
  https://gitlab.com/BuildGrid/buildgrid/issues/144
  

• 47e59e5e
  by Martin Blanchard at 2018-12-10T11:30:51Z

  cmd_operation.py: Port to new client channel helpers
  
  https://gitlab.com/BuildGrid/buildgrid/issues/144
  

• 0c6d6913
  by Martin Blanchard at 2018-12-10T11:30:51Z

  tests/utils: Add a file reader helper
  
  https://gitlab.com/BuildGrid/buildgrid/issues/144
  

• c4021884
  by Martin Blanchard at 2018-12-10T11:30:51Z

  tests/utils/dummy.py: New (dummy) server helper
  
  https://gitlab.com/BuildGrid/buildgrid/issues/144
  

• 7cbd0b8f
  by Martin Blanchard at 2018-12-10T11:30:51Z

  tests/auth: Add client and sever interceptor tests
  
  https://gitlab.com/BuildGrid/buildgrid/issues/144
  

• 929f314e
  by Martin Blanchard at 2018-12-10T11:30:51Z

  .gitlab-ci.yml: Install auth. dependencies for testing
  
  https://gitlab.com/BuildGrid/buildgrid/issues/144
  

• 2dc0ab0f
  by Martin Blanchard at 2018-12-10T11:39:19Z

  _app/settings/parser.py: Allow using dashes is key names
  
  https://gitlab.com/BuildGrid/buildgrid/issues/147
  

• 49cde519
  by Martin Blanchard at 2018-12-10T11:39:19Z

  _app/settings: Use dashes in example configurations
  
  https://gitlab.com/BuildGrid/buildgrid/issues/147
  

• e8f73080
  by Martin Blanchard at 2018-12-10T11:39:19Z

  docs: Use dashes in documentation configurations
  
  https://gitlab.com/BuildGrid/buildgrid/issues/147
  

• db2b3d30
  by Martin Blanchard at 2018-12-10T11:39:19Z

  cmd_server.py: Ensure configuration file exists
  
  https://gitlab.com/BuildGrid/buildgrid/issues/147
  

• 14beca26
  by Martin Blanchard at 2018-12-10T12:20:38Z

  cmd_server.py: Parse authorization configuration
  
  https://gitlab.com/BuildGrid/buildgrid/issues/147
  

• b55ce53c
  by Martin Blanchard at 2018-12-10T12:20:39Z

  _app/settings: Add a reference configuration file
  
  https://gitlab.com/BuildGrid/buildgrid/issues/147
  

• eba38225
  by Martin Blanchard at 2018-12-10T12:20:39Z

  docs: Add reference to server configuration section
  
  https://gitlab.com/BuildGrid/buildgrid/issues/147
  

• 2d8dbc30
  by Martin Blanchard at 2018-12-10T12:31:53Z

  Prefer 'empty' instance by default
  

• 73c7fc95
  by Javier Jardón at 2018-12-10T17:18:07Z

  setup.py: require at least protobuf 3.6.1
  
  Fixes #148
  

• 0c8e5a4c
  by Martin Blanchard at 2018-12-11T09:22:50Z

  reference.yml: Add a monitoring configuration section
  
  https://gitlab.com/BuildGrid/buildgrid/issues/142
  

• 6ab06a17
  by Martin Blanchard at 2018-12-11T09:22:52Z

  server/instance.py: Handle monitoring configuration
  
  https://gitlab.com/BuildGrid/buildgrid/issues/142
  

• 5fb63a15
  by Martin Blanchard at 2018-12-11T09:22:52Z

  cmd_server.py: Parse monitoring configuration
  
  https://gitlab.com/BuildGrid/buildgrid/issues/142
  

• 6c7ea3b2
  by Martin Blanchard at 2018-12-11T09:22:52Z

  scheduler.py: Allow dynamic monitoring activate/deactivation
  
  https://gitlab.com/BuildGrid/buildgrid/issues/142
  

• 37d52020
  by Martin Blanchard at 2018-12-11T09:22:52Z

  server/instance.py: Active schedulers monitoring
  
  https://gitlab.com/BuildGrid/buildgrid/issues/142
  

• 2265a9dc
  by Martin Blanchard at 2018-12-11T09:22:52Z

  _monitoring.py: Expose a 'prints_records?' property
  
  https://gitlab.com/BuildGrid/buildgrid/issues/142
  

• 53bbd05e
  by Martin Blanchard at 2018-12-11T09:22:52Z

  server/instance.py: Do not print log-record if monitoring bus does
  
  https://gitlab.com/BuildGrid/buildgrid/issues/142
  

• 7f391d70
  by Martin Blanchard at 2018-12-11T10:11:18Z

  job.py: Add an integer priority attribute
  

• 191d1515
  by Martin Blanchard at 2018-12-11T10:17:02Z

  Keep track of peers watching jobs
  

• 489d51ed
  by Martin Blanchard at 2018-12-11T10:32:24Z

  scheduler.py: Do not duplicate jobs for same action
  

• 256c3939
  by Martin Blanchard at 2018-12-11T10:43:04Z

  Deplicate Operations before streaming them
  

• 4e182bee
  by Martin Blanchard at 2018-12-11T10:52:05Z

  Track operations by peer
  

30 changed files:

• .gitlab-ci.yml
• .pylintrc
• + COMMITTERS.md
• CONTRIBUTING.rst
• − MAINTAINERS
• buildgrid/_app/bots/buildbox.py
• buildgrid/_app/bots/dummy.py
• buildgrid/_app/bots/host.py
• buildgrid/_app/cli.py
• buildgrid/_app/commands/cmd_bot.py
• + buildgrid/_app/commands/cmd_capabilities.py
• buildgrid/_app/commands/cmd_cas.py
• buildgrid/_app/commands/cmd_execute.py
• buildgrid/_app/commands/cmd_operation.py
• buildgrid/_app/commands/cmd_server.py
• buildgrid/_app/settings/cas.yml
• buildgrid/_app/settings/default.yml
• buildgrid/_app/settings/parser.py
• + buildgrid/_app/settings/reference.yml
• buildgrid/_app/settings/remote-storage.yml
• buildgrid/_enums.py
• buildgrid/_exceptions.py
• + buildgrid/_protos/buildgrid/__init__.py
• + buildgrid/_protos/buildgrid/v2/__init__.py
• + buildgrid/_protos/buildgrid/v2/monitoring.proto
• + buildgrid/_protos/buildgrid/v2/monitoring_pb2.py
• + buildgrid/_protos/buildgrid/v2/monitoring_pb2_grpc.py
• buildgrid/bot/bot.py
• − buildgrid/bot/bot_session.py
• + buildgrid/bot/hardware/__init__.py

The diff was not included because it is too large.

—
View it on GitLab.
You're receiving this email because of your account on gitlab.com. If you'd like to receive fewer emails, you can adjust your notification settings.