[Notes] [Git][BuildStream/buildstream][valentindavid/remote_execution_configuration] 22 commits: Rename "tools" directory to "requirements"

[Valentin David <gitlab mg gitlab com>]

Title: GitLab

Valentin David pushed to branch valentindavid/remote_execution_configuration at BuildStream / buildstream

Commits:

• ebd965fe
  by Chandan Singh at 2019-01-04T15:21:35Z

  Rename "tools" directory to "requirements"
  
  This seems like a better name for the directory, as it more closely
  describes the purpose of its contents.
  

• bb712886
  by Chandan Singh at 2019-01-04T15:21:35Z

  Add Makefile to update requirements files
  

• b9792837
  by Chandan Singh at 2019-01-04T15:21:35Z

  CONTRIBUTING.rst: Add instructions to update requirements files
  
  Split the "The MANIFEST.in and setup.py" section in two:
  "Managing data files" and "Updating BuildStream's Python dependencies".
  
  Briefly explain the layout of `requirements` directory and add
  instructions to use the Makefile added in the last commit.
  

• 45692335
  by Tristan Van Berkom at 2019-01-04T15:50:14Z

  Merge branch 'chandan/update-requirements-one-liner' into 'master'
  
  Add Makefile to update requirements files
  
  See merge request BuildStream/buildstream!1035

• f42dcb54
  by Chandan Singh at 2019-01-04T17:16:43Z

  .gitlab-ci.yml: Remove prepare stage
  
  As we now run tests using `tox`, we don't need to worry about manually
  packing and unpacking BuildStream. So, we can remove the preapre stage
  entirely.
  
  Update `coverage` and nightly jobs to appropriately cope with this
  change. Both these jobs now install all runtime dependencies from
  requirements files.
  

• ee29f207
  by Chandan Singh at 2019-01-04T17:41:02Z

  Merge branch 'chandan/no-sdist' into 'master'
  
  .gitlab-ci.yml: Remove prepare stage
  
  See merge request BuildStream/buildstream!1037

• f122cfe8
  by Jürg Billeter at 2019-01-06T17:57:33Z

  _frontend/app.py: Use buildtree for interactive shell on build failure
  
  Fixes: e29aea36 ("Basic options for shell --build to use buildtrees")
  

• 7dfb85b3
  by Tristan Van Berkom at 2019-01-06T18:31:25Z

  Merge branch 'juerg/shell-buildtree' into 'master'
  
  _frontend/app.py: Use buildtree for interactive shell on build failure
  
  See merge request BuildStream/buildstream!1039

• 891fcb0e
  by Tristan Van Berkom at 2019-01-07T16:47:01Z

  Fix stack traces discovered with ^C forceful termination.
  
    * utils.py:_kill_process_tree(): Ignore NoSuchProcess errors
  
      These are caused because we issue SIGTERM, and if the process
      has not exited after a timeout, we kill it.
  
    * _scheduler/jobs/job.py: Stop handling NoSuchProcess errors here
      redundantly, they are already ignored.
  
  It seems that we were ignoring it after sleeping when terminating
  tasks from the scheduler... but we were not ignoring it when performing
  the same pattern in the `Plugin.call()` -> `utils._call()` path, so
  we were still getting these exceptions at termination time from host
  tool processes launched by source plugins.
  

• 5de42d43
  by Tristan Van Berkom at 2019-01-07T18:00:37Z

  Merge branch 'tristan/keyboard-interrupt-stack-trace' into 'master'
  
  Fix stack traces discovered with ^C forceful termination.
  
  See merge request BuildStream/buildstream!1043

• 059035b9
  by Tristan Van Berkom at 2019-01-07T18:02:00Z

  _scheduler/scheduler.py: Make _schedule_jobs() private
  
  This is not used anywhere outside of the Scheduler, currently
  only the Scheduler itself is allowed to queue a job at this level.
  
  If the highlevel business logic for automatic queueing of auxiliary
  jobs moves to another location, we can make this public again.
  

• b83d1b1f
  by Tristan Van Berkom at 2019-01-07T18:02:00Z

  _scheduler/scheduler.py: Only run one cache size job at a time
  
  When queuing the special cache management related cleanup and
  cache size jobs, we now treat these jobs as special and do the
  following:
  
    * Avoid queueing a cleanup/cache_size job if one is already queued
  
      We just drop redundantly queued jobs here.
  
    * Ensure that jobs of this type only run one at a time
  
      This could have been done with the Resources mechanics,
      however as these special jobs have the same properties and
      are basically owned by the Scheduler, it seemed more straight
      forward to handle the behaviors of these special jobs together.
  
  This fixes issue #753
  

• 16a8816f
  by Tristan Van Berkom at 2019-01-07T18:02:00Z

  Scheduler: Introduced JobStatus instead of simple success boolean
  
  This changes the deepest callback from when a Job completes to
  propagate a JobStatus value instead of a simple boolean, and updates
  all of the effected code paths which used to receive a boolean
  to now handle the JobStatus values.
  
  This further improves the situation for issue #753, as now we avoid
  queueing cache size jobs for pull jobs which are skipped.
  

• c2fc2a5e
  by Tristan Van Berkom at 2019-01-07T18:02:00Z

  _scheduler/jobs/job.py: Removed 'skipped' property
  
  This is redundant now that we report it through the JobStatus.
  

• 3e3984ad
  by Tristan Van Berkom at 2019-01-07T18:50:23Z

  Merge branch 'tristan/one-cache-size-job' into 'master'
  
  Only queue one cache size job
  
  Closes #753
  
  See merge request BuildStream/buildstream!1040

• 512c726e
  by Tristan Van Berkom at 2019-01-08T03:38:11Z

  sandbox/sandbox.py: Fix regression of command logging
  
  Since we added batch commands, the batch commands print the
  text of the commands directly in the message text, but this is wrong.
  
  The detail string is the appropriate place for text of unknown lengths
  (the user can actually configure how many max lines of commands they
  want to see in their log), the message text itself should be controlled
  and brief enough to avoid text wrapping.
  

• 01171988
  by Tristan Van Berkom at 2019-01-08T04:20:14Z

  Merge branch 'tristan/fix-command-status-messages' into 'master'
  
  sandbox/sandbox.py: Fix regression of command logging
  
  See merge request BuildStream/buildstream!1044

• 6c1d06d6
  by Phil Dawson at 2019-01-08T10:24:32Z

  element.py: remove reference to source bundle command
  
  This command has been replacved by the bst source checkout command
  

• 914ecb72
  by Jürg Billeter at 2019-01-08T10:54:02Z

  Merge branch 'phil/remove-source-bundle-reference' into 'master'
  
  element.py: remove documentation reference to source bundle command
  
  See merge request BuildStream/buildstream!1041

• 04a9c781
  by Valentin David at 2019-01-08T11:30:49Z

  Use relative path to project directory for remote execution certificates/keys
  

• bff37b67
  by Valentin David at 2019-01-08T11:30:49Z

  Add support for user remote execution configuration
  
  Fixes #631.
  

• 3d5855bb
  by Valentin David at 2019-01-08T11:30:49Z

  Add support for https channel to remote execution and actions servers
  
  Fixes #780.
  

30 changed files:

• .gitlab-ci.yml
• CONTRIBUTING.rst
• MANIFEST.in
• buildstream/_context.py
• buildstream/_frontend/app.py
• buildstream/_project.py
• buildstream/_scheduler/__init__.py
• buildstream/_scheduler/jobs/__init__.py
• buildstream/_scheduler/jobs/cachesizejob.py
• buildstream/_scheduler/jobs/cleanupjob.py
• buildstream/_scheduler/jobs/elementjob.py
• buildstream/_scheduler/jobs/job.py
• buildstream/_scheduler/queues/buildqueue.py
• buildstream/_scheduler/queues/fetchqueue.py
• buildstream/_scheduler/queues/pullqueue.py
• buildstream/_scheduler/queues/queue.py
• buildstream/_scheduler/queues/trackqueue.py
• buildstream/_scheduler/scheduler.py
• buildstream/element.py
• buildstream/sandbox/_sandboxremote.py
• buildstream/sandbox/sandbox.py
• buildstream/utils.py
• doc/source/format_project.rst
• doc/source/using_config.rst
• + requirements/Makefile
• tools/dev-requirements.in → requirements/dev-requirements.in
• tools/dev-requirements.txt → requirements/dev-requirements.txt
• tools/plugin-requirements.in → requirements/plugin-requirements.in
• tools/plugin-requirements.txt → requirements/plugin-requirements.txt
• tools/requirements.in → requirements/requirements.in

The diff was not included because it is too large.

—
View it on GitLab.
You're receiving this email because of your account on gitlab.com. If you'd like to receive fewer emails, you can adjust your notification settings.