[Notes] [Git][BuildStream/buildstream][chandan/source-checkout] 115 commits: Fix some ruamel warnings during tests

[Chandan Singh <gitlab mg gitlab com>]

Title: GitLab

Chandan Singh pushed to branch chandan/source-checkout at BuildStream / buildstream

Commits:

• 2c6cb230
  by Valentin David at 2018-10-26T14:21:18Z

  Fix some ruamel warnings during tests
  

• a8250ca4
  by Valentin David at 2018-10-26T15:04:25Z

  Merge branch 'valentindavid/ruamel_warnings' into 'master'
  
  Fix some ruamel warnings during tests
  
  See merge request BuildStream/buildstream!889

• 1b308fe3
  by Daniel Silverstone at 2018-10-26T15:50:42Z

  _stream.py: Preserve stdout FD across checkout-to-stdout
  
  In order to support things which might need to use the stdout FD
  after checkout has completed writing a tarball to it, preserve it
  by means of `os.dup()` and `os.dup2()`
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 65d90c31
  by Daniel Silverstone at 2018-10-26T15:50:54Z

  tests/testutils/runcli.py: Support binary-mode capture of stdout
  
  In order to test things which write tarballs to stdout correctly,
  we need to capture the binary output cleanly.  This ensures we're not
  potentially mismatching encodings in and out.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• ed733f3e
  by Daniel Silverstone at 2018-10-26T15:51:40Z

  tests/frontend/buildcheckout.py: Enable test_build_checkout_tarball_stdout
  
  With the new support for saving the stdout FDs cleanly, and binary mode
  capture in runcli, enable the tarball-to-stdout test.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• e97d2a75
  by Benjamin Schubert at 2018-10-29T10:17:07Z

  Merge branch 'bschubert/fix-binary-output-capture' into 'master'
  
  Don't close stdout when running 'checkout' to stdout
  
  Closes #486
  
  See merge request BuildStream/buildstream!905

• 1625de0e
  by Tristan Van Berkom at 2018-10-29T11:49:33Z

  doc/source/conf.py: Added `images` directory for statically included files
  

• 38507b4e
  by Tristan Van Berkom at 2018-10-29T11:49:33Z

  doc/source/format_intro.rst: Adding link anchor for include directives
  

• 26164bcd
  by Tristan Van Berkom at 2018-10-29T13:57:51Z

  MANIFEST.in: Include SVG and ODG files in source distributions
  

• 3a23c8f6
  by Tristan Van Berkom at 2018-10-29T13:57:51Z

  doc: Adding new architecture document with initial "Overview of modules" section
  

• 41d370f6
  by Tristan Van Berkom at 2018-10-29T13:57:51Z

  doc: Adding new architecture document describing the data model
  

• bed7f7ac
  by Tristan Van Berkom at 2018-10-29T13:57:51Z

  doc: Adding new architecture document describing the dependency model
  

• 9bdf7f40
  by Tristan Van Berkom at 2018-10-29T13:57:51Z

  doc: Adding new architecture document about highlevel program flow
  

• 6b8bdf35
  by Tristan Van Berkom at 2018-10-29T13:57:51Z

  doc: Adding new architecture document about how the scheduler works.
  

• b1fadaa5
  by Tristan Van Berkom at 2018-10-29T13:57:51Z

  doc: Moved cachekey and sandboxing writings into the architecture section
  

• baf6b578
  by Tristan Van Berkom at 2018-10-29T14:28:41Z

  Merge branch 'tristan/architecture-docs' into 'master'
  
  Tristan/architecture docs
  
  See merge request BuildStream/buildstream!907

• 09ef8b25
  by Jonathan Maw at 2018-10-30T15:42:05Z

  element.py: Always clean up the rootdir
  
  We shouldn't need it to persist now that we cache failed build dirs.
  
  This change breaks the test
  `tests/integration/shell.py::test_sysroot_workspace_visible`.
  I can no longer see a use-case for this test.
  AIUI, it tested that the failed build sysroot stored in the builddir has
  the workspace's files in, despite the workspace being unmounted.
  I believe this behaviour is made redundant by cached buildtrees.
  
  This fixes part of #539
  

• 37afde0a
  by Jonathan Maw at 2018-10-30T15:42:05Z

  Element: Use cached buildtree in build shells and failure shells
  
  This includes changes in app.py:
  * Interactive failure shell no longer uses the failed build sysroot,
    defaulting to the cached build tree.
  
  Changes in element.py are:
  * Errors caused by building don't store the failed build sysroot,
    instead storing that a sandbox can be created to debug the error.
  * When staging sources, will stage the element's cached build tree if it
    exists.
  
  Changes in _exceptions.py:
  * BstError.sandbox is now a flag of whether a sandbox can be opened up
    to debug the error.
  
  Changes in widget.py:
  * Don't try to print any information about the sandbox.
  
  Changes in _message.py:
  * Fix documentation so Message.sandbox is not a directory any more.
  
  This is part of #539
  

• 4bc71773
  by Jonathan Maw at 2018-10-30T15:42:05Z

  NEWS: Add item for cached buildtree behaviour
  

• 9b89564f
  by Jonathan Maw at 2018-10-30T15:42:05Z

  tests: Add test that cached build trees are staged in build shells
  
  This is related to #539
  

• ec76cbe1
  by Jonathan Maw at 2018-10-30T15:42:05Z

  sandbox.py: Remove redundant Sandbox.__directory
  

• 825963b5
  by Jonathan Maw at 2018-10-30T15:42:05Z

  element: Make "--sysroot" take a bare directory
  
  i.e. instead of taking a directory that must contain "root" and
  "scratch", and treating "root" as the root, use the directory directly.
  
  In element.py:
  * __sandbox takes the `bare_sandbox` arg, to pass into the sandbox's constructor
  
  In sandbox.py:
  * If bare_sandbox, `_root` is the passed-in directory, and `__scratch`
    is None.
  * Trying to use `__scratch` when bare_sandbox is True is a bug.
  
  In _mount.py:
  * Don't get the value of `__scratch` if it's not needed.
  
  This is part of #539
  

• cf721838
  by Jonathan Maw at 2018-10-30T15:42:05Z

  NEWS: Add item for bst shell --sysroot changes
  

• 79d1bb7f
  by Jonathan Maw at 2018-10-30T15:42:05Z

  tests: Add tests for 'bst shell --sysroot'
  
  This is related to #539
  

• c7ac7e7d
  by Jonathan Maw at 2018-10-30T16:19:17Z

  Merge branch 'jonathan/debug-remote-failed-builds' into 'master'
  
  Jonathan/debug remote failed builds
  
  See merge request BuildStream/buildstream!869

• 516e990e
  by ctolentino8 at 2018-10-31T11:36:46Z

  _frontend/app.py: Set correct element-path in interactive bst-init
  

• b8a37a63
  by Tristan Van Berkom at 2018-11-01T10:16:25Z

  Merge branch 'chiaratolentino/bst-init-interactive-elementpath' into 'master'
  
  _frontend/app.py: Set correct element-path in interactive bst-init
  
  See merge request BuildStream/buildstream!912

• b27b592a
  by Benjamin Schubert at 2018-11-01T10:49:57Z

  Remove dependency on pytest-runner
  
  This includes a new command mimicking pytest-runner so that we
  can drop this dependency
  
  This was the only setup_requires dependency that we had and
  will make like easier for people behind proxies
  

• 89ace5d7
  by Benjamin Schubert at 2018-11-01T11:16:36Z

  Merge branch 'bschubert/remove-pytest-runner' into 'master'
  
  Remove dependency on pytest-runner
  
  See merge request BuildStream/buildstream!886

• 4cfabce8
  by Angelos Evripiotis at 2018-11-01T11:35:02Z

  BREAK: manual.yaml: don't set any default env vars
  
  Remove the 'MAKEFLAGS' and 'V' defaults.
  
  Now that there is a specialised 'make' element, these make-specific
  defaults don't make sense here. This element is meant to be a blank
  slate for folks to build on.
  
  Note that this is a breaking change, that will require some users to
  make changes to their .bst files if they are expecting these environment
  variable to be set.
  
  _versions.py: BST_FORMAT_VERSION bumped to 18.
  
  This fixes issue #718
  

• 48860aac
  by Tristan Van Berkom at 2018-11-01T12:01:04Z

  Merge branch 'tristan/empty-manual-element' into 'master'
  
  BREAK: manual.yaml: don't set any default env vars
  
  Closes #718
  
  See merge request BuildStream/buildstream!916

• d868b409
  by Daniel Silverstone at 2018-11-01T13:40:24Z

  _yaml.py: Implement `get()` for `ChainMap`
  
  Since the core Python `ChainMap.get()` implements with:
  
      self[key] if key in self else default
  
  The double-chain-lookup is expensive.  This simple change solves that
  for our ChainMap derived structure.  As such it improves matters
  for #466 somewhat.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 7f79b9ce
  by Tristan Van Berkom at 2018-11-01T14:25:57Z

  Merge branch 'danielsilverstone-ct/improve-chainmap-get' into 'master'
  
  _yaml.py: Implement `get()` for `ChainMap`
  
  See merge request BuildStream/buildstream!919

• de59ebdb
  by ctolentino8 at 2018-11-02T16:41:54Z

  plugins/sources/pip.py: Accomodate characters '-','.','_' for packages
  

• 8d7cf806
  by ctolentino8 at 2018-11-02T16:41:54Z

  tests/integration/project/files: Remove pypi-repo
  

• 9c2f9bf7
  by Chandan Singh at 2018-11-02T17:09:46Z

  Merge branch 'chiaratolentino/fix-pip-source-regex' into 'master'
  
  plugins/sources/pip.py: Accomodate characters '-','.','_' for packages
  
  See merge request BuildStream/buildstream!914

• 3788e701
  by Jürg Billeter at 2018-11-03T11:52:00Z

  element.py: Pass bare_directory also to SandboxRemote()
  
  Fixes: 825963b5 ("element: Make "--sysroot" take a bare directory")
  

• 82e971ef
  by Jürg Billeter at 2018-11-05T11:33:20Z

  Merge branch 'juerg/remote-execution' into 'master'
  
  element.py: Pass bare_directory also to SandboxRemote()
  
  See merge request BuildStream/buildstream!928

• 62942bfd
  by Valentin David at 2018-11-05T12:14:20Z

  Use less fetchers in overnight build to work-around limitations from servers.
  
  Issue happens with sourceware.org server.
  
  Fixes #742
  

• 442da2f9
  by Javier Jardón at 2018-11-05T12:41:54Z

  Merge branch 'valentindavid/limit_fetchers_in_overnight_build' into 'master'
  
  Use less fetchers in overnight build to work-around limitations from servers.
  
  Closes #742
  
  See merge request BuildStream/buildstream!929

• 0993de53
  by Richard Maw at 2018-11-05T16:41:05Z

  dev-requirements: Update pytest dependency to 3.8
  
  "_frontend/app.py: Set correct element-path in interactive bst-init"
  added a dependency on the tmp_path test fixture, which was added by 3.8.
  

• be8f0a54
  by richardmaw-codethink at 2018-11-05T17:08:43Z

  Merge branch 'richardmaw/pytest-dep-update' into 'master'
  
  dev-requirements: Update pytest dependency to 3.8
  
  See merge request BuildStream/buildstream!930

• bfb639bf
  by Jürg Billeter at 2018-11-05T17:18:12Z

  element.py: Remove artifacts parameter from constructor
  
  Get the artifact cache from the context instead.
  

• ca855f91
  by Jürg Billeter at 2018-11-05T17:18:12Z

  _elementfactory.py: Remove unused artifacts parameter from create()
  

• 15fed21c
  by Jürg Billeter at 2018-11-05T17:18:12Z

  _project.py: Remove unused artifacts parameter from create_element()
  

• 0085d2aa
  by Jürg Billeter at 2018-11-05T17:18:12Z

  element.py: Remove unused artifacts parameter from _new_from_meta()
  

• f69b1117
  by Jürg Billeter at 2018-11-05T17:18:12Z

  _project.py: Remove unused artifacts parameter from load_elements()
  

• e398f877
  by Jürg Billeter at 2018-11-05T17:18:12Z

  element.py: Do not include type name of artifact cache in cache key
  
  The artifact cache backend does not affect build outputs and we anyway
  no longer have pluggable artifact cache backends. This hardcodes
  CASCache instead of removing the entry completely to avoid cache key
  changes.
  

• 626d20ae
  by Jürg Billeter at 2018-11-05T17:18:12Z

  Split up artifact cache and CAS cache
  
  This changes CASCache from a subclass to a delegate object of
  ArtifactCache. As the lower layer, CASCache no longer deals with
  elements or projects.
  
  Fixes #659.
  

• ec04446b
  by Jürg Billeter at 2018-11-05T17:51:39Z

  Merge branch 'juerg/cas' into 'master'
  
  Split up artifact cache and CAS cache
  
  Closes #659
  
  See merge request BuildStream/buildstream!922

• 9a045080
  by Josh Smith at 2018-11-06T13:23:19Z

  utils: Fix _pretty_size() for sizes > 1024T
  

• d3a07e6b
  by Josh Smith at 2018-11-06T13:23:19Z

  Add regression test for _pretty_size
  

• 0c09fb9c
  by richardmaw-codethink at 2018-11-06T14:12:20Z

  Merge branch 'Qinusty/unit-test-utils' into 'master'
  
  Fix issue with _pretty_size with large numbers of bytes
  
  See merge request BuildStream/buildstream!799

• b4eec489
  by Jim MacArthur at 2018-11-06T16:44:14Z

  _sandboxremote.py: Use the standard SandboxError.
  
  Replaces the custom one which was erroneously added during development.
  Fixes #746.
  

• 1f7acf74
  by Jim MacArthur at 2018-11-06T17:16:31Z

  Merge branch 'jmac/remote-sandbox-errors' into 'master'
  
  _sandboxremote.py: Use the standard SandboxError
  
  Closes #746
  
  See merge request BuildStream/buildstream!931

• 8e64ccef
  by James Ennis at 2018-11-07T11:06:11Z

  using_config.rst: Update our instructions for how we declare caches in user config
  

• 09ab676d
  by James Ennis at 2018-11-07T11:06:11Z

  format_project.rst: Update instructions regarding how we declare remote caches in project.conf
  

• f514124f
  by James Ennis at 2018-11-07T11:06:11Z

  using_configuring_artifact_server.rst: Remove examples and redirect to user config/project config
  

• 029ba17d
  by richardmaw-codethink at 2018-11-07T11:32:20Z

  Merge branch 'jennis/docs_patch' into 'master'
  
  Update our instructions/examples regarding how we declare remote caches
  
  See merge request BuildStream/buildstream!932

• 261e2cd3
  by Jim MacArthur at 2018-11-07T12:02:43Z

  element.py: Use _force_rmtree instead of custom code.
  
  There are arguments for and against using onerror - see issue #735
  for details.
  

• 8931e42c
  by Jim MacArthur at 2018-11-07T12:26:06Z

  Merge branch 'jmac/make_writable_combination' into 'master'
  
  Use _force_rmtree instead of custom code.
  
  Closes #735
  
  See merge request BuildStream/buildstream!927

• 6ccfab0b
  by Valentin David at 2018-11-07T16:28:47Z

  Fix cache corruption by scripts when layout and integration commands are used
  
  Root directory was marked as a non-artifact mount, so not using
  SafeHardLink.  However integration commands executed with write access
  to the root directory.
  
  Fixes #749
  

• f24e20e9
  by richardmaw-codethink at 2018-11-07T16:59:50Z

  Merge branch 'valentindavid/script-artifact-corruption' into 'master'
  
  Fix cache corruption by scripts when layout and integration commands are used
  
  Closes #749
  
  See merge request BuildStream/buildstream!934

• e28f668a
  by Valentin David at 2018-11-08T09:34:11Z

  Fix infinite recursion in default strip debug command
  
  Fixes #645.
  

• 83d15350
  by Valentin David at 2018-11-08T10:01:39Z

  Merge branch 'valentindavid/fix-recursive-strip-debug' into 'master'
  
  Fix infinite recursion in default strip debug command
  
  Closes #645
  
  See merge request BuildStream/buildstream!918

• 9f0e12f1
  by Benjamin Schubert at 2018-11-08T10:21:12Z

  Move bwrap checks in platform/linux.py
  
  Remove the bwraps checks from _site.py and put them in platform.linux
  which is the only place where they are run.
  
  This allows the removal of a double level of caching, making
  reasoning about tests easier
  

• cf2e0059
  by Benjamin Schubert at 2018-11-08T10:21:12Z

  conftest.py: Ensure platform is not maintained between tests
  
  This removes the `_instance` on the platform object that we use
  for caching and not recreating the object everytime at the start
  of every test.
  
  This is to ensure our tests share the least amount of state.
  
  The performance penalty is from 5 to 10% accross the whole test suite.
  The readings were done 5 times for each before and after the change
  and on the same computer.
  

• c51ba01b
  by Benjamin Schubert at 2018-11-08T10:21:12Z

  Test that helpful messages are raised when missing dependencies
  
  This adds a `reason` to the SandboxEror thrown in sandboxdummy
  to be able to understand where the error comes from
  

• f116b9b7
  by Benjamin Schubert at 2018-11-08T11:07:12Z

  Merge branch 'bschubert/tests-no-chroot-linux' into 'master'
  
  test behavior when there is no sandbox on Linux
  
  Closes #696 and #736
  
  See merge request BuildStream/buildstream!921

• cc2e6ae5
  by Valentin David at 2018-11-08T12:41:36Z

  Fix bug with root mounted as non-artifact in script plugin.
  
  The issue was introduced by 6ccfab0b1b25990e406446d5cbe5aee83a5e158a.
  

• e578a89f
  by Valentin David at 2018-11-08T13:11:10Z

  Merge branch 'valentindavid/fix-script-install-root' into 'master'
  
  Fix bug with root mounted as non-artifact in script plugin.
  
  Closes #756 and #749
  
  See merge request BuildStream/buildstream!941

• c0a8bb66
  by Angelos Evripiotis at 2018-11-08T15:49:16Z

  contributing: WIP again to be kind to reviewers
  
  As someone coming from GitHub to GitLab, I was pleasantly surprised by
  the 'filter by WIP status' option.
  
  To make sure we get the most out of it, add a guideline to keep the
  filter clean, to reduce reviewer burden.
  

• f7231e90
  by Angelos Evripiotis at 2018-11-08T15:49:16Z

  contributing: non-WIP should always be landable
  
  I was recently surprised that we don't prefer addressing review comments
  in 'fixup!' commits.
  
  Coming from GitHub, I've found that fixup commits make it easier for
  reviewers to see what has changed since their last review. The idea is
  to use '--autosquash' before landing to clean up the history again.
  
  It's a pleasant surprise that it's easy to keep track of what changed in
  merge-requests between pushes, so we can always keep the history clean.
  
  Document this, so folks like me can see the light sooner.
  

• f7643440
  by Angelos Evripiotis at 2018-11-08T15:49:16Z

  contributing: emphasise the 'why' in commits
  
  Explain why it's useful to mention the decisions made for a change and
  provide links for background, e.g. issue numbers.
  
  My general experience of git histories is that folks don't do enough
  explaining, so it's great we have a contributing section for it.
  
  I've found it's easier for folks to stick to rules when they know the
  practical reasons for them, so provide some.
  

• dd5e7b04
  by Angelos Evripiotis at 2018-11-08T16:17:04Z

  Merge branch 'aevri/contributing_gitlab' into 'master'
  
  Add more to GitLab-relevant parts of contributing
  
  See merge request BuildStream/buildstream!935

• fe33e328
  by James Ennis at 2018-11-08T17:54:18Z

  using_config.rst: Add documentation to showing how to impose quotas on the local cache
  
  This patch partially resolves #700
  

• 09faf002
  by James Ennis at 2018-11-08T17:54:18Z

  artifactcache.py: Fix misleading error message when using % cache quota
  
  Due to the changed Exception message, this patch also changes
  the test_parse_size_over_1024T test in misc.py
  

• d153453c
  by Javier Jardón at 2018-11-08T18:22:54Z

  Merge branch 'jennis/quota_declaration_fix' into 'master'
  
  Add local cache expiry documentation and fix misleading error message when specifying a percentage cache quota
  
  Closes #700
  
  See merge request BuildStream/buildstream!939

• 7738f6df
  by Daniel Silverstone at 2018-11-09T11:25:44Z

  plugin.py: Redirect DEBUG from `__del__` to `sys.stderr`
  
  Previously, debug messages from `Plugin.__del__()` went to stdout
  which was a problem for the test suite in extremely rare and hard to
  predict circumstances.  This corrects that by sending the message to
  stderr where all the rest of the messages tend to go.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 62f59eaa
  by Valentin David at 2018-11-09T13:30:06Z

  Merge branch 'danielsilverstone-ct/plugin-destroy-debug-to-stderr' into 'master'
  
  plugin.py: Redirect DEBUG from `__del__` to `sys.stderr`
  
  See merge request BuildStream/buildstream!944

• 35ec6b6a
  by Jim MacArthur at 2018-11-09T14:58:22Z

  _casbaseddirectory.py: Enable direct CAS-to-CAS import.
  
  Fixes issue #574.
  

• b93100ec
  by Jim MacArthur at 2018-11-09T14:58:22Z

  _casbaseddirectory: Fix order of list_modified_paths
  
  This will now return paths in the same order as list_relative_paths.
  

• 78691fa0
  by Jim MacArthur at 2018-11-09T14:58:22Z

  virtual_directory_import.py: New test
  

• 7ce6581b
  by Jim MacArthur at 2018-11-09T15:25:46Z

  Merge branch 'jmac/cas_to_cas_v2' into 'master'
  
  Direct CAS-to-CAS import
  
  Closes #574
  
  See merge request BuildStream/buildstream!911

• 6f4351ec
  by Benjamin Schubert at 2018-11-09T16:53:42Z

  source.py: don't let StopIteration propagate to silence() contextmanager
  
  As per PEP 0479 (https://www.python.org/dev/peps/pep-0479/),
  StopIteration thrown in context managers are not valid starting from
  Python 3.7.
  

• e6e03451
  by richardmaw-codethink at 2018-11-12T11:05:15Z

  Merge branch 'bschubert/fix-silence-stopiteration-pep-0479' into 'master'
  
  source.py: don't let StopIteration propagate to silence() contextmanager
  
  See merge request BuildStream/buildstream!945

• ae5ccd76
  by Phillip Smyth at 2018-11-12T14:57:51Z

  _project.py: Validate nodes early in Project._load
  
  _project.py: Added validate_nodes() helper function to prevent duplicate lists
  
  element-path was being used before node validation resulting in uncaught errors
  

• 56a54161
  by Phillip Smyth at 2018-11-12T14:57:51Z

  tests/frontend: Add test for invalid element-path
  

• e55a9703
  by Jürg Billeter at 2018-11-12T15:44:45Z

  Merge branch 'element-path_not_validated' into 'master'
  
  Element path not validated before use
  
  See merge request BuildStream/buildstream!937

• c87d6e45
  by Valentin David at 2018-11-14T11:43:33Z

  Run tests on aarch64
  
  Fixes #755
  

• 16b01489
  by Valentin David at 2018-11-14T11:43:33Z

  Disable cachekey tests on other architectures than x86_64
  

• 85046b29
  by Valentin David at 2018-11-14T11:43:33Z

  Disable tests on example on other architectures than x86_64
  

• dd5a073b
  by Valentin David at 2018-11-14T11:43:33Z

  Add support for aarch64 in a test
  

• c5d72ae7
  by Valentin David at 2018-11-14T11:52:53Z

  Test aarch64 only overnight
  

• 264a57f6
  by Javier Jardón at 2018-11-14T12:22:03Z

  Merge branch 'valentindavid/integration-tests-aarch64' into 'master'
  
  Run tests on aarch64
  
  Closes #369 and #755
  
  See merge request BuildStream/buildstream!948

• da735e56
  by Richard Maw at 2018-11-14T13:30:34Z

  _platform/linux.py: Move get_bwrap_version into _site.py
  
  It's inconvenient to have to create a Linux platform to parse the bwrap version
  and we want to get the version in a consistent manner.
  

• e7633500
  by Richard Maw at 2018-11-14T13:30:34Z

  buildstream/sandbox/_sandboxbwrap.py: Distinguish sandbox failure from command failure
  
  If `bwrap` fails to set up the sandbox and start the payload command
  it won't write an exit-code in --json-status-fd,
  so we can report if it was a sandboxing failure if we don't get exit-code status
  and a payload command failure if we do and it's non-zero.
  
  Closes https://gitlab.com/BuildStream/buildstream/issues/286
  

• e9e08823
  by Richard Maw at 2018-11-14T13:30:34Z

  tests/testutils/site.py: Check for bwrap supporting --json-status-fd
  

• 90ca007e
  by Richard Maw at 2018-11-14T13:30:34Z

  tests/integration/sandbox-bwrap.py: Test distinguishing sandbox exit code from command
  

• 327b19dd
  by richardmaw-codethink at 2018-11-14T13:59:16Z

  Merge branch 'richardmaw/distinguish-sandboxing-build-fail' into 'master'
  
  Distinguish between bubblewrap sandboxing failure and command failure
  
  Closes #286
  
  See merge request BuildStream/buildstream!868

• 082b3811
  by Angelos Evripiotis at 2018-11-15T19:48:12Z

  contributing: add guidance on unit tests
  
  Decrease uncertainty around whether unit tests are welcome in the
  project or not.
  

• b5b79056
  by Jürg Billeter at 2018-11-15T20:24:58Z

  Merge branch 'aevri/unit_tests' into 'master'
  
  contributing: add guidance on unit tests
  
  See merge request BuildStream/buildstream!943

• 199bfff1
  by Tom Pollard at 2018-11-17T13:07:59Z

  Add cli main and user config option for 'pull-buildtrees' context.
  
  _context.py: Add pull_buildtrees global user context, the default
  of which is set to False via the addition of pull-buildtrees to
  userconfig.yaml cache group.
  
  _frontend/app.py & cli.py: Add --pull-buildtrees as a bst main
  option, which when passed will override the default or user defined
  context for pull_buildtrees.
  
  tests/completions/completions.py: Update for the added flag.
  

• 454dcaf1
  by Tom Pollard at 2018-11-17T13:07:59Z

  Don't pull artifact build trees by default.
  
  The addition of cached build trees being included in element
  artifacts has led to mostly redundant download overheads when
  pulling from a remote artifact server. As such the default behaviour
  of pull shouldn't fetch the build tree object if available.
  
  element.py: extend relevant pull logic for specified subdir
  consideration and ensure push logic does not lead to partial
  artifact pushes. Change assumption that the buildtree can be
  extracted if artifact is cached. __cached_buildtree() and
  __pull_directories() helpers added.
  
  _artifactcache/: artifactcache.py & cascache.py inclusion of
  helper functions for subdir artifact checking & fetching, fetch
  logic extended to only pull required artifact directories.
  extract & checkout updated to handle full/partial operation.
  
  tests/: addition of integration test pullbuildtrees.py,
  buildtrees.py adapted cli options, testutils/artifactshare.py
  has_artifact changed to return artifact digest on true condition.
  

• 0d309478
  by Jürg Billeter at 2018-11-17T13:49:58Z

  Merge branch 'tpollard/494' into 'master'
  
  Don't pull artifact buildtrees by default
  
  See merge request BuildStream/buildstream!786

• 27932739
  by Benjamin Schubert at 2018-11-19T09:46:39Z

  plugins/sources/pip.py: also look for python version named "python"
  
  In some virtual environments, we might not have the binary with the
  exact python version, which would lead the current pip plugin to
  fail to find a working pip version.
  
  This adds "python" at the start of the list of valid python versions
  

• ea2de561
  by Benjamin Schubert at 2018-11-19T10:19:24Z

  Merge branch 'bschubert/fix-pip-python' into 'master'
  
  plugins/sources/pip.py: also look for python version named "python"
  
  Closes #758
  
  See merge request BuildStream/buildstream!942

• f23b6031
  by Benjamin Schubert at 2018-11-19T11:39:51Z

  tests/frontend/buildtrack.py: check for success after invocation
  
  Previously the result of the first call in test_build_track was not
  checked to be successful, which would lead to potential erros later on
  that were hard to debug.
  
  Let's check it everytime
  

• a6defc0b
  by Benjamin Schubert at 2018-11-19T11:39:51Z

  Fix os.rename in git source element to correctly handle error codes
  
  According to the documentation
  (https://www.unix.com/man-page/POSIX/3posix/rename/), when the directory
  already is there, either EEXIST or ENOTEMPTY could be thrown.
  
  Previously only ENOTEMPTY was checked.
  
  Done:
    - Separated the move into its own function
    - Check for both errors
    - Create unit tests for it, covering most test cases
  

• 88089d2d
  by Benjamin Schubert at 2018-11-19T11:39:51Z

  Extract atomic move function to utils.py
  
  Moving atomically a file/directory can be tricky since different
  errors might be raised for the same underlying problem.
  
  Having a utility function to reduce this discrepancies will help
  in ensuring we have correct behavior
  

• fd9e46be
  by Benjamin Schubert at 2018-11-19T11:39:51Z

  cascache.py: use move_atomic instead of manual error checking
  

• d32e0b83
  by Benjamin Schubert at 2018-11-19T11:39:51Z

  sources/pip.py: Use move_atomic instead of manual os.rename
  
  This uses move_atomic insteand of the manual os.rename and manual error
  checking and throws a SourceError for consistency with other modules.
  

• 6f837118
  by Benjamin Schubert at 2018-11-19T12:22:40Z

  Merge branch 'bschubert/fix-atomic-move-git-repo' into 'master'
  
  Fix os.rename in git source element to correctly handle error codes
  
  See merge request BuildStream/buildstream!938

• 30b72244
  by Chandan Singh at 2018-11-19T12:35:01Z

  buildstream/_pipeline.py: Simplify error message for assert_consistent()
  
  Currently, `Pipeline.assert_consistent()` prints an error message like
  the one below when certain elements have inconsistent sources:
  
  ```
  ...
        Element: hello.bst is inconsistent
          Source tar source at hello.bst [line 16 column 2] is missing ref
  ...
  ```
  
  Drop the word "Source" from the beginning of the message as
  "Source <source-kind> source ..." is kind of awkward to read.
  

• 76c5d2f8
  by Chandan Singh at 2018-11-19T12:35:01Z

  Add `bst source-checkout` command
  
  As discussed in
  https://mail.gnome.org/archives/buildstream-list/2018-September/msg00064.html,
  add `bst source-checkout` command. This will allow users to checkout
  sources for a given target.
  
  * _frontend/cli.py: Add source-checkout command
  * _pipeline.py: Add assert_sources_cached() method
  * _stream.py: Add source_checkout method, abstract out
    __check_location_writable() method that used to part of checkout()
  

30 changed files:

• .gitlab-ci.yml
• CONTRIBUTING.rst
• MANIFEST.in
• NEWS
• buildstream/_artifactcache/artifactcache.py
• buildstream/_artifactcache/cascache.py
• buildstream/_artifactcache/casserver.py
• buildstream/_context.py
• buildstream/_elementfactory.py
• buildstream/_exceptions.py
• buildstream/_frontend/app.py
• buildstream/_frontend/cli.py
• buildstream/_frontend/widget.py
• buildstream/_loader/loader.py
• buildstream/_message.py
• buildstream/_pipeline.py
• buildstream/_platform/linux.py
• buildstream/_project.py
• buildstream/_site.py
• buildstream/_stream.py
• buildstream/_versions.py
• buildstream/_yaml.py
• buildstream/data/projectconfig.yaml
• buildstream/data/userconfig.yaml
• buildstream/element.py
• buildstream/plugin.py
• buildstream/plugins/elements/manual.yaml
• buildstream/plugins/sources/git.py
• buildstream/plugins/sources/pip.py
• buildstream/sandbox/_mount.py

The diff was not included because it is too large.

—
View it on GitLab.
You're receiving this email because of your account on gitlab.com. If you'd like to receive fewer emails, you can adjust your notification settings.