[Notes] [Git][BuildStream/buildstream][Qinusty/634-workspace-failed-builds] 95 commits: Fix rst links for guideline links

[Tristan Maat <gitlab mg gitlab com>]

Title: GitLab

Tristan Maat pushed to branch Qinusty/634-workspace-failed-builds at BuildStream / buildstream

Commits:

• 335d5a45
  by Adam Jones at 2018-09-19T13:00:45Z

  Fix rst links for guideline links
  

• 667dd4ca
  by Javier Jardón at 2018-09-19T13:25:36Z

  Merge branch 'adamjones/contributing-links' into 'master'
  
  Fix rst link formatting for guideline links
  
  See merge request BuildStream/buildstream!811

• 68ef69e4
  by Tristan Van Berkom at 2018-09-21T05:20:46Z

  source.py: Fix re-instantiation
  
  This fixes an issue where the re-instantiated Source used with
  Source mirroring enabled is not completely initialized.
  
  Failing to load the ref from the project.refs file for instance,
  will result in a crash at `fetch` time.
  
  This fixes issue #666
  

• 662c729f
  by Tristan Van Berkom at 2018-09-21T05:59:30Z

  Merge branch 'tristan/fix-source-reinstantiation' into 'master'
  
  source.py: Fix re-instantiation
  
  Closes #666
  
  See merge request BuildStream/buildstream!817

• 461a0588
  by Jim MacArthur at 2018-09-21T10:53:11Z

  element.py: Remove early warning and check directory is not None
  
  The check for directory should have always been here. The early warning
  test doesn't work for `bst checkout` because caches haven't been configured
  at the point of the test, so I've had to remove it.
  

• aa9caaac
  by Jim MacArthur at 2018-09-21T10:53:11Z

  _sandboxremote.py: Ignore return value of push_directory
  
  The return value is always upload_vdir.ref.
  

• 2aae68c7
  by Jim MacArthur at 2018-09-21T10:53:11Z

  cascache.py: make push_remote raise exception if no push remotes exist
  
  Also add docstring to method and remove return value since it was useless
  

• ca1bb72c
  by Jim MacArthur at 2018-09-21T10:53:11Z

  tests/artifactcache/push.py: Ignore return value of push_directory
  

• 55c93a82
  by Jim MacArthur at 2018-09-21T11:26:55Z

  Merge branch 'jmac/remote_exec_checkout_fix' into 'master'
  
  Remote exec: Remove early warning and check directory is not None
  
  See merge request BuildStream/buildstream!800

• e209beb0
  by Chandan Singh at 2018-09-21T13:10:08Z

  _stream.py: Ensure source-bundle's source directory exists
  
  Currently, `source-bundle` command is entirely broken as it tries to stage the
  sources in a directory that doesn't exist. Fix it by ensuring that we create
  the necessary directories before calling any methods that try to use those
  directories.
  
  This fix comes with a regression test to ensure that the basic use-case
  of `source-bundle` continues to work in future.
  
  Fixes https://gitlab.com/BuildStream/buildstream/issues/651.
  

• 0b000518
  by Chandan Singh at 2018-09-21T13:56:55Z

  Merge branch 'chandan/fix-source-bundle' into 'master'
  
  Fix source-bundle command
  
  Closes #651
  
  See merge request BuildStream/buildstream!807

• ef26043a
  by Chandan Singh at 2018-09-21T17:14:16Z

  setup.py: Make website the primary homepage
  

• 1b2aed40
  by Chandan Singh at 2018-09-21T17:40:11Z

  Merge branch 'chandan/update-project-homepage' into 'master'
  
  setup.py: Make website the primary homepage
  
  See merge request BuildStream/buildstream!795

• da7e038b
  by Jürg Billeter at 2018-09-24T13:58:55Z

  element.py: Fix cache check in non-strict mode
  
  The behavior of __assert_cached() was changed when the keystrength
  parameter was introduced. This restores the previous behavior.
  
  Fixes #607.
  
  Fixes: 3d308894 ("element.py: Cache the result of checking whether...")
  

• 86ea1173
  by Jürg Billeter at 2018-09-24T13:59:59Z

  tests/frontend/rebuild.py: Add strict/non-strict rebuild test
  
  Test rebuild that requires staging of weakly cached dependency.
  
  This is a regression test for #607.
  

• a76339de
  by Jürg Billeter at 2018-09-24T16:02:38Z

  Merge branch 'juerg/rebuild' into 'master'
  
  element.py: Fix cache check in non-strict mode
  
  Closes #607
  
  See merge request BuildStream/buildstream!822

• b199afe6
  by Jürg Billeter at 2018-09-25T09:01:51Z

  _artifactcache/cascache.py: Add _ensure_blob helper
  
  This adds directory objects to the local repository before downloading
  files in the directory. However, artifact references are still stored
  only after downloading the complete directory and thus, there won't be
  dangling references. This will anyway be required for partial download
  support.
  

• 7d199322
  by Jürg Billeter at 2018-09-25T09:01:51Z

  _artifactcache/cascache.py: Increase payload size limit for uploads
  
  gRPC can handle 1 MiB payloads. Increase size limit from 64 KiB to speed
  up uploads.`
  

• e2e24015
  by Jürg Billeter at 2018-09-25T09:01:51Z

  _artifactcache/casserver.py: Harmonize payload size limit
  
  Use 1 MiB as payload size limit on the server side for both individual
  downloads and batch uploads.
  

• 697d10f2
  by Jürg Billeter at 2018-09-25T09:01:51Z

  _artifactcache/cascache.py: Use BatchReadBlobs
  
  This uses BatchReadBlobs instead of individual blob download to speed up
  artifact pulling, if the server supports it.
  
  Fixes #554.
  

• 81c51dbf
  by Jürg Billeter at 2018-09-25T09:31:55Z

  Merge branch 'juerg/cas-batch' into 'master'
  
  _artifactcache/cascache.py: Use BatchReadBlobs
  
  Closes #554
  
  See merge request BuildStream/buildstream!813

• 2df7d140
  by Jim MacArthur at 2018-09-25T10:36:37Z

  sandbox.py: Stop caching get_virtual_directory if get_directory is used
  
  Also update the documentation for get_virtual_directory to make the
  implications of using both methods clearer.
  

• 62f59382
  by Jim MacArthur at 2018-09-25T10:36:37Z

  element.py/_assemble(): Call get_virtual_directory later.
  

• 8cea7b17
  by Jim MacArthur at 2018-09-25T10:58:40Z

  Merge branch 'jmac/stop-caching-vdirs' into 'master'
  
  Stop caching virtual directories if get_directory is used.
  
  See merge request BuildStream/buildstream!818

• 83ab183e
  by Tiago Gomes at 2018-09-25T16:37:30Z

  CI: test building freedesktop-sdk overnight
  
  Closes #514.
  

• c8594806
  by Tiago Gomes at 2018-09-25T17:03:55Z

  Merge branch 'tiagogomes/issue-514' into 'master'
  
  CI: test building freedesktop-sdk overnight
  
  Closes #514
  
  See merge request BuildStream/buildstream!826

• fdbf9be1
  by Tiago Gomes at 2018-09-26T08:17:46Z

  ci: update freedesktop-sdk ref
  
  The current one doesn't build due server being down.
  

• 9280b0cf
  by Tiago Gomes at 2018-09-26T15:54:53Z

  ci: add ACME challenge
  
  To finish the creation of the Let's Encrypt certificate, which will
  allows us to use https://docs.buildstream.build/ for accessing the
  documentation.
  

• af131503
  by Tiago Gomes at 2018-09-26T16:35:58Z

  Merge branch 'tiagogomes/acme-challenge' into 'master'
  
  ci: add ACME challenge
  
  See merge request BuildStream/buildstream!831

• f7f14f25
  by Tiago Gomes at 2018-09-26T17:42:12Z

  ci: add missing `-p` argument
  

• 0f2bc375
  by Martin Blanchard at 2018-09-27T08:46:27Z

  _protos: Import google.rpc.Code proto
  
  Imports code.proto from https://github.com/googleapis/googleapis and
  corresponding modules generated with ./setup.py build_grpc.
  
  https://gitlab.com/BuildStream/buildstream/issues/668
  

• 16cf9d5f
  by Martin Blanchard at 2018-09-27T08:46:27Z

  _sandboxremote.py: Handle remote build execution exit code
  
  https://gitlab.com/BuildStream/buildstream/issues/668
  

• f5f3cb7c
  by Jim MacArthur at 2018-09-27T09:16:38Z

  Merge branch 'mablanch/668-remote-build-failure' into 'master'
  
  Better handle remote build failures
  
  See merge request BuildStream/buildstream!825

• b99a6740
  by Jürg Billeter at 2018-09-27T09:18:03Z

  element.py: Add __use_remote_execution() helper method
  

• f8bbe008
  by Jürg Billeter at 2018-09-27T09:18:03Z

  _platform: Add check_sandbox_config() method
  

• f9494f1f
  by Jürg Billeter at 2018-09-27T09:18:03Z

  element.py: Validate sandbox config
  

• b4ad84c3
  by Jürg Billeter at 2018-09-27T09:18:03Z

  _platform/linux.py: Remove redundant message
  
  Unsupported sandbox config (UID/GID) is now reported by the element.
  

• 966af3d2
  by Jürg Billeter at 2018-09-27T09:18:03Z

  _platform/linux.py: Do not disable push globally
  
  On systems without user namespace support, elements with unsupported
  sandbox config (UID/GID) are now individually tainted, which disables
  artifact push.
  

• f08e5eae
  by Jürg Billeter at 2018-09-27T09:18:03Z

  _artifactcache/cascache.py: Remove unused enable_push parameter
  

• 10ed9158
  by Jürg Billeter at 2018-09-27T09:18:03Z

  _artifactcache: Add preflight() method
  

• 7aec8b0f
  by Jürg Billeter at 2018-09-27T09:18:03Z

  Move artifact cache creation to Context
  
  The artifact cache is no longer platform-specific.
  

• a5025c33
  by Jürg Billeter at 2018-09-27T09:18:03Z

  _loader: Migrate to Context.artifactcache
  

• ef1cb374
  by Jürg Billeter at 2018-09-27T09:18:03Z

  _scheduler: Migrate to Context.artifactcache
  

• 52991be1
  by Jürg Billeter at 2018-09-27T09:18:03Z

  _stream.py: Migrate to Context.artifactcache
  

• cb6b35dc
  by Jürg Billeter at 2018-09-27T09:18:42Z

  sandbox/_sandboxremote.py: Migrate to Context.artifactcache
  

• 28c62b00
  by Jürg Billeter at 2018-09-27T09:18:43Z

  _platform/platform.py: Remove unused artifactcache property
  

• 99e90918
  by Jürg Billeter at 2018-09-27T09:19:12Z

  _artifactcache/casserver.py: Use Context for artifact cache creation
  

• b8f920e2
  by Jürg Billeter at 2018-09-27T09:19:12Z

  storage/_casbaseddirectory.py: Use Context for artifact cache creation
  

• 3d88c5d4
  by Jürg Billeter at 2018-09-27T09:19:12Z

  tests/testutils: Use Context for artifact cache creation
  

• ed878cb6
  by Jürg Billeter at 2018-09-27T09:19:12Z

  tests/artifactcache/pull.py: Use Context for artifact cache creation
  

• fe82c9c2
  by Jürg Billeter at 2018-09-27T09:19:12Z

  tests/artifactcache/push.py: Use Context for artifact cache creation
  

• 0d0f700d
  by Jürg Billeter at 2018-09-27T09:19:12Z

  _platform: Remove unused context
  

• dd770ec3
  by Jürg Billeter at 2018-09-27T09:19:12Z

  Use lazy platform instantiation
  
  Now that the platform is independent of the context, explicit
  instantiation is no longer required. This avoids issues with platform
  instances used across test cases with mismatching context.
  

• 44da8175
  by Jürg Billeter at 2018-09-27T09:48:20Z

  Merge branch 'juerg/platform' into 'master'
  
  Make platform independent of context
  
  See merge request BuildStream/buildstream!829

• 900e8900
  by Jürg Billeter at 2018-09-27T13:12:34Z

  git.py: Make `ref` human readable
  
  Use `git describe` to prefix `ref` with closest tag, if available.
  

• 107269c1
  by Jürg Billeter at 2018-09-27T13:12:34Z

  testutils/repo/git.py: Add add_tag() method
  

• 8060ad8c
  by Jürg Billeter at 2018-09-27T13:12:34Z

  tests/sources/git.py: Add track and fetch test with and without tag
  

• ab1cb672
  by Jürg Billeter at 2018-09-27T13:46:41Z

  Merge branch 'juerg/git-describe' into 'master'
  
  git.py: Make `ref` human readable
  
  See merge request BuildStream/buildstream!291

• fcf37f9c
  by knownexus at 2018-09-27T14:22:09Z

  Added `set_resource_limits()` to platform
  
  This has been moved from app.py
  As it will have different functionality depending on platform
  Once the Darwin (MacOS) platform is added
  
  Removed `resource.setrlimit()` functionality from app.py
  Added `resource.setrlimit()` functionality to platform.py as function
  

• 80ebdd2c
  by knownexus at 2018-09-27T14:22:09Z

  Added FUSE check, as WSL does not support FUSE yet
  
  _platform/linux.py: Add FUSE check
  And Ensure exception is not raised when performing sandbox availability check
  Without this check, WSL would error upon finding FUSE missing
  This must be bypassed to allow remote execution
  

• 5e81573b
  by knownexus at 2018-09-27T14:22:09Z

  Max-jobs is now defined on a platform basis
  
  This is due to Darwin (MacOS) having a different Max-Jobs limit
  _platform/platform.py: Adding max-jobs call
  
  _project.py: redirected 'max-jobs' to point at the platform
  

• 3c93fe97
  by knownexus at 2018-09-27T14:22:09Z

  Adding Dummy sandbox
  
  This is to allow platforms that do not support sandboxing
  To error in a controlled/known way
  

• 0f3ef369
  by knownexus at 2018-09-27T14:22:24Z

  Adding darwin.py (MacOS) platform
  
  Adding functionality to recognise Darwin as a platform in plaform.py
  

• b78ae767
  by knownexus at 2018-09-27T14:29:29Z

  utils.py: Reworked safe_remove
  
  Non-Linux platforms don't return EISDIR when attempting to unlink a
  directory.
  
  Stopped safe_remove attempting to unlink dirs
  Previously safe_remove would attempt to unlink a path
  Before attempting to remove it if it was a dir
  
  Now it checks for a dir before that step
  

• ab5e78b4
  by James Ennis at 2018-09-27T14:29:32Z

  Import sandbox subclass when instantiating sandbox
  
  This patch was required due to not being able to execute `bst` on
  a Linux machine without FUSE available.
  

• 261f65ca
  by Jürg Billeter at 2018-09-27T14:54:47Z

  Merge branch 'mac_fixes' into 'master'
  
  Implement compatibility fixes for MacOSX and WSL Blocks #411 and #412"
  
  See merge request BuildStream/buildstream!726

• d2ea3069
  by Richard Maw at 2018-09-27T15:16:57Z

  tests: Add regression test for pushing failed builds
  
  This test ensures that failed builds are pushed when set to on-error=quit.
  
  Closes https://gitlab.com/BuildStream/buildstream/issues/534
  

• c128429a
  by richardmaw-codethink at 2018-09-27T15:42:18Z

  Merge branch 'richardmaw/push-failed-build-regression' into 'master'
  
  tests: Add regression test for pushing failed builds
  
  Closes #534
  
  See merge request BuildStream/buildstream!824

• e91eb38b
  by Valentin David at 2018-09-27T16:32:57Z

  Fix outside-of-project check when project path is not canonical.
  
  The issue happens on Silverblue where /home is a symlink to /var/home.
  With this element-path is something like
  /var/home/user/project/elements, when the project path is
  /home/usr/project. Comparing canonical paths solves the issue.
  
  Fixes #673
  

• 7babd55c
  by Tiago Gomes at 2018-09-27T16:57:12Z

  Merge branch 'valentindavid/silverblue-path' into 'master'
  
  Fix outside-of-project check when project path is not canonical.
  
  Closes #673
  
  See merge request BuildStream/buildstream!832

• 5cdfac81
  by Mathieu Bridon at 2018-09-28T09:20:20Z

  data/userconfig.yaml: Document the cache.quota setting
  

• 5d3f039f
  by Tom Pollard at 2018-09-28T10:34:13Z

  Merge branch 'bochecha/cache-size' into 'master'
  
  data/userconfig.yaml: Document the cache.quota setting
  
  See merge request BuildStream/buildstream!833

• 14e1a3b3
  by Jürg Billeter at 2018-10-01T08:18:05Z

  _platform: Fix get_cpu_count() with cap=None
  
  Comparison between None and integer is not supported.
  

• 232662f1
  by Jürg Billeter at 2018-10-01T08:53:06Z

  Merge branch 'juerg/cpucount' into 'master'
  
  _platform: Fix get_cpu_count() with cap=None
  
  See merge request BuildStream/buildstream!840

• f447aedd
  by Tiago Gomes at 2018-10-01T10:33:11Z

  ci: upper case variables
  
  For consistency.
  

• 682dddce
  by Tiago Gomes at 2018-10-01T10:35:12Z

  ci: correct freedestop-sdk version comment
  

• fafa8136
  by Tiago Gomes at 2018-10-01T10:59:54Z

  Merge branch 'tiagogomes/ci-minor-tweaks' into 'master'
  
  Minor CI tweaks
  
  See merge request BuildStream/buildstream!842

• 26e1a3c7
  by Jürg Billeter at 2018-10-01T14:58:06Z

  _artifactcache/casserver.py: Implement BatchUpdateBlobs
  
  Fixes #676.
  

• f47895c0
  by Jürg Billeter at 2018-10-01T14:58:06Z

  _artifactcache/cascache.py: Use BatchUpdateBlobs
  
  This uses BatchUpdateBlobs instead of individual blob upload to speed up
  artifact pushing, if the server supports it.
  
  Fixes #677.
  

• cf00c0a1
  by Jürg Billeter at 2018-10-01T15:32:30Z

  Merge branch 'juerg/cas-batch' into 'master'
  
  CAS: Implement BatchUpdateBlobs support
  
  Closes #677 and #676
  
  See merge request BuildStream/buildstream!839

• 5f4ae90b
  by Jürg Billeter at 2018-10-02T06:34:02Z

  sandbox/_sandboxremote.py: Use helpers to set up cwd and env
  
  This matches the other sandbox implementations.
  

• 0458bc4e
  by Jürg Billeter at 2018-10-02T07:08:35Z

  Merge branch 'juerg/sandbox' into 'master'
  
  sandbox/_sandboxremote.py: Use helpers to set up cwd and env
  
  See merge request BuildStream/buildstream!841

• d5b396e0
  by Phillip Smyth at 2018-10-02T16:40:18Z

  Forcing CI to install Sphinx 1.7.9
  
  .gitlab-ci.yml: forced pip3 install Sphinx 1.7.9
  The theme we use in our docs "sphinx_rtd_theme"
  Does not support the latest version of Sphinx (>1.8)
  This results in search functionality failing
  

• dae842fd
  by Phillip Smyth at 2018-10-02T17:14:09Z

  Merge branch 'docs_Search_not_working' into 'master'
  
  Fixing: Search functionality in the Documentation has stopped working
  
  Closes #681
  
  See merge request BuildStream/buildstream!848

• 8dc16d3f
  by Jürg Billeter at 2018-10-03T05:08:21Z

  _platform/linux.py: Accept all configs for dummy sandbox
  
  If Sandbox.run() is never called, sandbox config is irrelevant. If it is
  called, the build will fail anyway with the dummy sandbox.
  
  This matches the logic in the Darwin platform.
  

• 66446fc3
  by Jürg Billeter at 2018-10-03T05:36:38Z

  Merge branch 'juerg/dummy-sandbox' into 'master'
  
  _platform/linux.py: Accept all configs for dummy sandbox
  
  See merge request BuildStream/buildstream!843

• 29c19bea
  by Tristan Van Berkom at 2018-10-03T07:33:48Z

  git.py: Added timed activity around fetching, displaying the URL being fetched.
  
  This will take care of silencing the status messages while checking submodules.
  

• b645881c
  by Tristan Van Berkom at 2018-10-03T07:33:48Z

  source.py: Silence messages while consuming source fetchers
  
  The source fetchers might be a list or a generator, when it is
  a generator (like the git source does), then we want to ensure that
  we silence the status messages which might occur as a result of
  consuming a source fetcher from the generator.
  
  This fixes the logs to be less verbose.
  

• c9437616
  by Tristan Van Berkom at 2018-10-03T08:07:15Z

  Merge branch 'tristan/fix-status-messages' into 'master'
  
  fix status messages
  
  See merge request BuildStream/buildstream!845

• 11320fe2
  by Tristan Van Berkom at 2018-10-03T09:33:39Z

  _scheduler: Fix bookkeeping of terminated jobs
  
   * Enhanced the base Job class to bookkeep which jobs have
     been terminated, and to consider them as `skipped` when
     asked via the `skipped` property.
  
   * Enhanced the base Queue class to bookkeep the job statuses
     more carefully.
  
  This fixes #479
  

• 91271964
  by Tristan Van Berkom at 2018-10-03T09:59:40Z

  Merge branch 'tristan/fix-terminated-jobs' into 'master'
  
  _scheduler: Fix bookkeeping of terminated jobs
  
  Closes #479
  
  See merge request BuildStream/buildstream!850

• 3bf895d2
  by Jonathan Maw at 2018-10-03T11:48:48Z

  tests: Add regression test for mirroring with project.refs
  

• e4969807
  by Jonathan Maw at 2018-10-03T12:48:07Z

  Merge branch 'jonathan/source-mirror-project-refs' into 'master'
  
  tests: Add regression test for mirroring with project.refs
  
  See merge request BuildStream/buildstream!823

• e6d37335
  by Josh Smith at 2018-10-03T13:03:25Z

  element.py: Do not save workspace on failed build
  
  This fixes #634, BuildStream no longer saves workspace configuration on
  a failed build.
  

• a88396f4
  by Josh Smith at 2018-10-03T13:03:25Z

  testing: Add regression test for workspace bug
  

30 changed files:

• .gitlab-ci.yml
• CONTRIBUTING.rst
• buildstream/_artifactcache/artifactcache.py
• buildstream/_artifactcache/cascache.py
• buildstream/_artifactcache/casserver.py
• buildstream/_context.py
• buildstream/_frontend/app.py
• buildstream/_loader/loader.py
• + buildstream/_platform/darwin.py
• buildstream/_platform/linux.py
• buildstream/_platform/platform.py
• buildstream/_platform/unix.py
• buildstream/_project.py
• + buildstream/_protos/google/rpc/code.proto
• + buildstream/_protos/google/rpc/code_pb2.py
• + buildstream/_protos/google/rpc/code_pb2_grpc.py
• buildstream/_scheduler/jobs/cachesizejob.py
• buildstream/_scheduler/jobs/cleanupjob.py
• buildstream/_scheduler/jobs/job.py
• buildstream/_scheduler/queues/buildqueue.py
• buildstream/_scheduler/queues/queue.py
• buildstream/_scheduler/scheduler.py
• buildstream/_stream.py
• buildstream/_yaml.py
• buildstream/data/userconfig.yaml
• buildstream/element.py
• buildstream/plugins/sources/git.py
• buildstream/sandbox/__init__.py
• + buildstream/sandbox/_sandboxdummy.py
• buildstream/sandbox/_sandboxremote.py

The diff was not included because it is too large.

—
View it on GitLab.
You're receiving this email because of your account on gitlab.com. If you'd like to receive fewer emails, you can adjust your notification settings.