[Notes] [Git][BuildStream/buildstream][jmac/make_marked_directories] 199 commits: _pipeline.py: Fix the planner to retain the original order when depth sorting

[Jim MacArthur <gitlab mg gitlab com>]

Title: GitLab

Jim MacArthur pushed to branch jmac/make_marked_directories at BuildStream / buildstream

Commits:

• 0b83d024
  by Tristan Van Berkom at 2019-01-10T20:02:50Z

  _pipeline.py: Fix the planner to retain the original order when depth sorting
  
  The algorithm adds elements to a dictionary and then sorts the dictionary
  by the discovered depths while recursing - using an OrderedDict is enough
  to ensure a stable order.
  
  This fixes `bst show --deps plan ...` reporting different results on
  each invocation.
  
  This fixes an aspect of #824
  

• 630e26f1
  by Tristan Van Berkom at 2019-01-10T20:03:52Z

  testutils/runcli.py: Added Result.get_start_order()
  
  Added a function to report the list of elements which appeared in
  a given queue in the order of their first appearance.
  

• d1d7de57
  by Tristan Van Berkom at 2019-01-11T04:13:16Z

  tests/frontend/order.py: Adding tests ensuring stable element processing order
  
  For each sample project and expected result order, this test ensures that:
  
    * bst show --deps plan: Always shows the expected build order.
  
    * bst source fetch: Always executes in the expected build order.
  
    * bst build: Always builds in the expected order (disabled).
  
  The build part of the test is currently disabled as the scheduler
  seems to be messing with the order even in a `--builders 1` scenario.
  

• 276b8d48
  by Tristan Van Berkom at 2019-01-11T04:42:24Z

  Merge branch 'tristan/element-processing-order' into 'master'
  
  Make build plan element list stable
  
  See merge request BuildStream/buildstream!1058

• fc3dcec8
  by Angelos Evripiotis at 2019-01-11T16:45:00Z

  dev-requirements: pytest-cov==2.6.1 for '--no-cov'
  
  Bump the version of pytest-cov, so that we pick up the fix for
  the '--no-cov' option breaking in pytest 4.0:
  
      https://github.com/pytest-dev/pytest-cov/pull/230
  
  This was the error you would get prior to this version:
  
      _pytest.warning_types.RemovedInPytest4Warning: config.warn has been
      deprecated, use warnings.warn instead
  
  Although it says 'deprecated', it did result in
  termination with stack trace.
  

• f86b7ff3
  by Angelos Evripiotis at 2019-01-11T16:45:00Z

  contributing: '--last-failed' and '--no-cov' tips
  
  To help new folks get to grips with pytest, add some tips on other
  frequently used features.
  

• d983f231
  by Angelos Evripiotis at 2019-01-11T17:19:06Z

  Merge branch 'aevri/tox_no_cov' into 'master'
  
  pytest-cov==2.6.1, contributing: '--last-failed' and '--no-cov' tips
  
  See merge request BuildStream/buildstream!1059

• 829a2b93
  by Tristan Van Berkom at 2019-01-12T21:32:13Z

  _scheduler/queues/queue.py: Put elements in the skip list, not jobs
  
  Minor correction, looks like we're not observing this queue
  otherwise we'd be seeing crashes.
  

• 993e30ae
  by Tristan Van Berkom at 2019-01-12T22:02:55Z

  Merge branch 'tristan/fix-skipped-elements' into 'master'
  
  _scheduler/queues/queue.py: Put elements in the skip list, not jobs
  
  See merge request BuildStream/buildstream!1063

• 32732e01
  by Javier Jardón at 2019-01-14T09:04:01Z

  .gitlab-ci.yml: Use latest freedesktop-sdk 18.08.25
  

• 4f5f1184
  by Valentin David at 2019-01-14T09:40:56Z

  Merge branch 'jjardon/freedesktop-sdk_latest' into 'master'
  
  .gitlab-ci.yml: Use latest freedesktop-sdk 18.08.25
  
  Closes #858
  
  See merge request BuildStream/buildstream!1064

• bb80a2b8
  by Chandan Singh at 2019-01-14T14:30:15Z

  .gitlab-ci.yml: Use newer image for overning aarch tests
  
  During recent reorganizing of `.gitlab-ci.yml` to work with `tox`, seems
  like we missed to update the image used by the overnight aarch tests,
  meaning that they currently fail due to `tox` being missing from them.
  
  While these tests will be skipped on MRs usually, here is an example of
  what the tests look like if they are actually run with this change:
  https://gitlab.com/BuildStream/buildstream/-/jobs/145449561.
  
  Fixes #859.
  

• 10b3ee62
  by Chandan Singh at 2019-01-14T14:57:08Z

  Merge branch 'chandan/fix-overnight-aarch' into 'master'
  
  Chandan/fix overnight aarch
  
  Closes #859
  
  See merge request BuildStream/buildstream!1065

• 542cdaf0
  by Jürg Billeter at 2019-01-14T18:46:57Z

  _frontend/cli.py: Also check original args for --config in bash completion
  
  The path of the config file generated by testutils for completion tests
  is passed as regular argument, not via COMP_WORDS. Use that config file
  in complete_artifact() to ensure the test uses the right artifact
  directory.
  

• ff666e76
  by James Ennis at 2019-01-14T18:46:57Z

  completions.py: Add a test for our artifact ref autocompletions
  
  Since the artifact subcommand group has been created, we have been able
  to tab complete both element names and artifact refs as arguments to
  the artifact subcommands (e.g. bst artifact log), this commit adds a
  test for this.
  

• 273b0f55
  by Tristan Van Berkom at 2019-01-14T20:24:57Z

  Merge branch 'jennis/add_artifacts_completion' into 'master'
  
  completions.py: Add a test for our artifact ref autocompletions
  
  See merge request BuildStream/buildstream!1054

• 2e3c2ea2
  by Javier Jardón at 2019-01-14T22:26:04Z

  .gitlab-ci.yml: Use latest bst-external
  
  which include flatpak_repo plugin needed to build fdsdk
  

• d60d2e31
  by Javier Jardón at 2019-01-15T00:39:22Z

  Merge branch 'jjardon/bst_external_0_9' into 'master'
  
  .gitlab-ci.yml: Use latest bst-external
  
  Closes #861
  
  See merge request BuildStream/buildstream!1068

• 605836c1
  by Chandan Singh at 2019-01-15T00:57:05Z

  .gitlab-ci.yml: Add job to attempt to update dependencies
  
  This will help us check if BuildStream is working with the latest
  version of dependencies, as per our constraints. This job is allowed to
  fail but its failure should signal that we need to add stricter
  constraints in some of our `.in` requirements files.
  

• 006370af
  by Chandan Singh at 2019-01-15T01:21:33Z

  Merge branch 'chandan/reqs-update-test' into 'master'
  
  .gitlab-ci.yml: Add job to attempt to update dependencies
  
  See merge request BuildStream/buildstream!1038

• 4b544555
  by Chandan Singh at 2019-01-15T21:28:40Z

  .gitlab-ci.yml: Add tests for python 3.7
  
  We already have tests for python 3.5 and 3.6 but not 3.7.
  
  Fixes https://gitlab.com/BuildStream/buildstream/issues/838.
  

• 80fe0d9a
  by Javier Jardón at 2019-01-15T22:48:22Z

  Merge branch 'chandan/python37-tests' into 'master'
  
  .gitlab-ci.yml: Add tests for python 3.7
  
  Closes #838
  
  See merge request BuildStream/buildstream!1074

• c91784ab
  by Chandan Singh at 2019-01-15T22:49:01Z

  conftest.py: Don't use deprecated get_marker() function
  
  Starting from `pytest` version 4.1.0, `Node.get_marker()` has been
  removed, and hence our tests break when running with newer versions of
  `pytest`. It was deprecated since a while back but it has recently been
  removed completely. Use `get_closest_marker()` as a replacement that is
  suggested in the changelog, and seems to work fine for our use case.
  
  See https://github.com/pytest-dev/pytest/pull/4564 for more context on
  the upstream issue.
  
  One way of verifying this change is that this should fix the recently
  added `tests-fedora-update-deps` job, that was failing before due to
  this issue.
  

• ecae4d73
  by Javier Jardón at 2019-01-15T23:23:41Z

  Merge branch 'chandan/fix-pytest-get-marker' into 'master'
  
  conftest.py: Don't use deprecated get_marker() function
  
  See merge request BuildStream/buildstream!1073

• 0eac4008
  by Valentin David at 2019-01-16T10:04:57Z

  buildstream/_gitsourcebase.py: Reduce git history for git describe.
  
  Found during #833.
  
  `git rev-list --boundary tag..HEAD` unfortunately gives boundaries
  that are deeper than should when there is a merge commit between `tag`
  and `HEAD`. The common ancestory of the two parents of the merge is
  a boundary instead of the parent of the branch that is not traversed.
  
  `--ancestry-path` fixes this issue by restricting `git` traversing
  those branches.
  

• a405e08f
  by Valentin David at 2019-01-16T10:04:57Z

  buildstream/_gitsourcebase.py: Fix case where HEAD is tagged
  
  `git rev-list --boundary HEAD..HEAD` does not return any boundary.
  So in this case we need to manually tag the HEAD as a boundary.
  

• c0631d48
  by Valentin David at 2019-01-16T10:33:52Z

  Merge branch 'valentindavid/git-reduced-history' into 'master'
  
  buildstream/_gitsourcebase.py: Reduce git history for git describe.
  
  See merge request BuildStream/buildstream!1069

• b608ac86
  by James Ennis at 2019-01-16T11:05:44Z

  _context.py: Doc fix: get_toplevel_project() returns object not list
  

• 97a3beb6
  by James Ennis at 2019-01-16T11:05:44Z

  _context.py: Add documentation to get_workspaces() command
  

• d6587aa0
  by James Ennis at 2019-01-16T11:33:21Z

  Merge branch 'jennis/doc_fixes_in_context' into 'master'
  
  Small documentation/comment fixes in context.py
  
  See merge request BuildStream/buildstream!1072

• 2683f98a
  by Raoul Hidalgo Charman at 2019-01-16T11:55:07Z

  _cas: Rename artifactcache folder and move that to a root module
  
  Other components will start to reply on cas modules, and not the artifact cache
  modules so it should be organized to reflect this.
  
  All relevant imports have been changed.
  
  Part #802
  

• d2cc4798
  by Raoul Hidalgo Charman at 2019-01-16T11:55:07Z

  casremote.py: Move remote CAS classes into its own file
  
  Part of #802
  

• 76f67483
  by Raoul Hidalgo Charman at 2019-01-16T11:55:07Z

  cas: move remote only functions to CASRemote
  
  List of methods moved
  * Initialization check: made it a class method that is run in a subprocess, for
    when checking in the main buildstream process.
  * fetch_blobs
  * send_blobs
  * verify_digest_on_remote
  * push_method
  
  Part of #802
  

• 6c428bc9
  by Jürg Billeter at 2019-01-16T12:56:38Z

  Merge branch 'raoul/cas-refactor' into 'master'
  
  Cas refactor
  
  See merge request BuildStream/buildstream!1071

• 2648b0bd
  by Angelos Evripiotis at 2019-01-16T14:44:25Z

  bst-docker-import: fix Chandan's name

• 780d0f3a
  by Angelos Evripiotis at 2019-01-16T15:13:06Z

  Merge branch 'chadnan' into 'master'
  
  bst-docker-import: fix Chandan's name
  
  See merge request BuildStream/buildstream!1076

• 4ca37dcd
  by Tristan Van Berkom at 2019-01-16T15:47:21Z

  source-determinism.py integration test: Dont use integration_cache fixture
  
  This was just deadcode, it's not really used here.
  

• 6286d820
  by Tristan Van Berkom at 2019-01-16T15:47:21Z

  conftest.py: Use different artifact directory for integration tests
  
  To ensure we can run integration tests in parallel, use a tempdir
  for the artifact cache of each separate integration test run.
  
  This makes it possible to run multiple full test runs including
  integration tests in parallel under detox.
  

• 471af316
  by Tristan Van Berkom at 2019-01-16T16:32:52Z

  Merge branch 'tristan/detoxing-integration-tests' into 'master'
  
  Make integration tests parallelizable
  
  See merge request BuildStream/buildstream!1077

• 081e8426
  by James Ennis at 2019-01-16T17:58:41Z

  setup.py: Import server main from _cas/casserver.py
  
  Since the CAS refactor, we have not been able to execute
  bst-artifact-server. This commit ensures that we can.
  
  This fix closes #867
  

• e385660c
  by James Ennis at 2019-01-16T18:33:20Z

  Merge branch 'jennis/fix_bst_artifact_server' into 'master'
  
  Ensure that we can execute `bst-artifact-server`
  
  Closes #867
  
  See merge request BuildStream/buildstream!1079

• 6991d6d6
  by Tristan Van Berkom at 2019-01-16T19:28:14Z

  _scheduler: Refactor of queues and resources.
  
  This branch makes the following changes:
  
    * jobs/job.py: No longer stores any interested resource list
  
      Jobs are ephemeral again, they only ever exist while they
      are running.
  
    * queues/queue.py: Revert to only handling lists of elements
  
      Elements pass through the queues, Queue.harvest_jobs()
      replaces Queue.pop_ready_jobs() and now the Queue stops
      creating jobs as soon as there are not enough resources
      for the job.
  
      Also removed unused `prepare()` abstract method.
  
    * queues/buildqueue.py: Adapt the part where we launch a job
  
      This part needs to be reworked anyway, just touch it up for
      now so that it doesnt break with the surrounding changes.
  
    * jobs/{cachesize,cleanup}job.py: Expose uniform complete callback
  
      Allows the scheduler to manage resource deallocation for these
      two job completions as a custom thing, at the same phase
      that the Queues take care of their own resource deallocation.
  
    * resources.py: No longer has knowledge of the job
  
      Since jobs are ephemeral, they are not a suitable place
      to store the resource identifiers, these must be provided
      by the callers wherever needed.
  
      Now the main Resources object is owned by the Scheduler
      but shared with Queues, each take care of managing the
      resources of the jobs they create through the same
      resource API.
  
    * scheduler.py: Reverted to only creating jobs on demand
  
      This changes the flow of the scheduler such that whenever
      jobs complete, the queues are interrogated for as many
      jobs which can run at the moment but not more; and this
      completely removes the waiting list.
  
      For the internal cache management jobs, we handle this
      with a little state instead of having a waiting list
      and only launch when the resources permit it.
  
  By abolishing the scheduler waiting list and creating jobs
  on demand, we fix the order of element processing and consequently
  fix issue #712.
  

• 04ba59bc
  by Tristan Van Berkom at 2019-01-16T19:28:14Z

  tests/frontend/order.py: Enable the test for build and fix the fetch tests
  
  With the scheduler changes, fetch jobs get automatically skipped
  so the output is changed, using a separate repo for each element
  fixes the test such that every fetch job gets a job launched.
  

• 32bdded8
  by Tristan Van Berkom at 2019-01-16T19:28:14Z

  _artifactcache/artifactcache.py: Rephrase failure message
  
  It was saying "There is not enough space to build the given element.",
  this makes me think the error is associated to a specific element, but
  this does not make sense to show up in a cleanup task.
  
  Instead say "There is not enough space to complete the build.", which
  should be more clear that even after cleaning up there is not enough
  space.
  

• 17f6e5a8
  by Tristan Van Berkom at 2019-01-16T20:10:30Z

  Merge branch 'tristan/element-processing-order' into 'master'
  
  Scheduler refactor, fix processing order
  
  Closes #712
  
  See merge request BuildStream/buildstream!1067

• 19adc9e5
  by Tristan Van Berkom at 2019-01-16T20:48:10Z

  element.py: Cleanup temporary staging directories on termination
  
  Use utils._tempdir() which in turn uses _signals.terminator() for
  this purpose.
  
  This also changes utils._tempdir() to use utils._force_rmtree()
  so that it is safe to use for cleaning up the staging directory.
  
  This patch fixes orphaned temporary directories being left behind
  in the artifact cache directory at forceful termination time.
  

• 5e9598c5
  by Tristan Van Berkom at 2019-01-16T21:17:26Z

  Merge branch 'tristan/cleanup-staging-tempdir' into 'master'
  
  element.py: Cleanup temporary staging directories on termination
  
  See merge request BuildStream/buildstream!1080

• 5ea2f59d
  by Tristan Van Berkom at 2019-01-16T23:35:21Z

  tests: Migrate completions test to tests/frontend
  

• 1eafc25c
  by Tristan Van Berkom at 2019-01-16T23:35:21Z

  tests: Migrate `--except` testing to tests/frontend/show.py
  
  The tests/pipeline directory will be removed, and this test
  is the better of the two but also redundant with the one in
  tests/frontend/show.
  
  Renamed existing test in show.py to `test_show_except_simple`
  and added the better test as `test_show_except` below it.
  

• b6183123
  by Tristan Van Berkom at 2019-01-16T23:35:21Z

  tests: Migrate virtual directory storage test to internals directory
  
  Grouping bits of internal testing together here
  

• 9b6cc972
  by Tristan Van Berkom at 2019-01-16T23:35:21Z

  tests: Migrate storage test to the internals directory
  

• 7e2300ce
  by Tristan Van Berkom at 2019-01-16T23:35:21Z

  tests: Migrate yaml test to the internals directory
  

• 353a682f
  by Tristan Van Berkom at 2019-01-16T23:35:21Z

  tests: Migrate utils tests into internals directory
  

• 3324490e
  by Tristan Van Berkom at 2019-01-16T23:35:21Z

  tests: Migrate context test into the internals directory
  

• 99ea157a
  by Tristan Van Berkom at 2019-01-16T23:35:21Z

  tests: Migrate plugin factory test to internals directory
  

• e825bdd3
  by Tristan Van Berkom at 2019-01-16T23:35:21Z

  tests: Migrate plugin loading test to internals directory
  

• c7f69bcc
  by Tristan Van Berkom at 2019-01-16T23:35:21Z

  tests: Removing tests/plugins/third_party.py
  
  This tests exactly the same thing that is tested in
  tests/internals/pluginfactory.py (the new location
  of tests/plugins/basics.py).
  

• 42150422
  by Tristan Van Berkom at 2019-01-16T23:35:21Z

  tests: Rename `plugins` directory to `elements` directory
  
  Now that the remaining test "filter.py" in the plugins directory
  tests a specific element, it makes sense to create a place for
  testing elements, just like we do for sources.
  

• f712aaa5
  by Tristan Van Berkom at 2019-01-16T23:35:21Z

  tests: Migrate internal Loader basic test to internals directory
  

• 8d676a84
  by Tristan Van Berkom at 2019-01-16T23:35:21Z

  tests: Migrate dependencies test to tests/format
  
  This used to be an internal test, converted this to use the `cli` fixture.
  

• bbde1c21
  by Tristan Van Berkom at 2019-01-16T23:35:21Z

  tests: Migrate dependency order/iteration testing to the format tests
  
  Created new `tests/format/iteration.py` which tests the order in
  which elements are iterated over in various scopes in a loaded
  data model.
  

• 8f2b4e9a
  by Tristan Van Berkom at 2019-01-16T23:35:21Z

  tests: Migrate test for load_ref() unsupporting plugins into format/project.py
  
  The tests/format/project.py test already has some tests about how
  we error gracefully for bad plugins and bad plugin configurations,
  lets put it there rather than tests/pipeline/load.py which we
  will remove.
  

• de881c2a
  by Tristan Van Berkom at 2019-01-16T23:35:21Z

  tests: Migrate preflight error handling check to tests/format/project.py
  
  This is where other load time related plugin error handling is
  checked, and is the last thing to remove in the `tests/pipeline`
  directory.
  

• a7fbd900
  by Tristan Van Berkom at 2019-01-16T23:35:21Z

  tests: Renaming some tests in tests/format/project.py
  
  Remove some redundancy from the test lines.
  

• aa997450
  by Tristan Van Berkom at 2019-01-16T23:35:21Z

  tests: Removing tests/pipeline/load.py
  
  The remaining test simply loads a project with one element
  and asserts a value on it. This is already sufficiently tested
  in tests/format/project.py.
  

• 561eddf1
  by Tristan Van Berkom at 2019-01-16T23:35:21Z

  tests: Migrate protected variable handling tests to tests/format/variables.py
  

• 3bf40cf3
  by Tristan Van Berkom at 2019-01-16T23:35:21Z

  tests: Migrate junctions test to tests/format/junctions.py
  
  This is the directory for all things related to loading.
  

• d34a4fd1
  by Tristan Van Berkom at 2019-01-17T00:26:43Z

  Merge branch 'tristan/organizing-tests' into 'master'
  
  General refactor in tests directory
  
  See merge request BuildStream/buildstream!1062

• d212cdfa
  by Tristan Van Berkom at 2019-01-17T00:28:03Z

  sandbox/sandbox.py: Display failed commands in the detail string
  
  We should only display commands in detail strings, not in the
  message texts.
  
  This also updates tests/integration/sandbox-bwrap.py to expect
  the new message string which only contains the command exit status
  and not the whole command itself, this does not alter the validity
  of the text case which is checking that we can obtain the expected
  return value.
  

• ce91ce5d
  by Tristan Van Berkom at 2019-01-17T05:05:16Z

  Merge branch 'tristan/error-message-regression' into 'master'
  
  sandbox/sandbox.py: Display failed commands in the detail string
  
  See merge request BuildStream/buildstream!1081

• 63b02f7c
  by James Ennis at 2019-01-17T15:13:27Z

  _profile.py: Add timestamp to the logs
  

• f64ee46f
  by James Ennis at 2019-01-17T17:02:28Z

  _profile.py: Write binaries as well as logs
  
  Private class methods which write the logs and write the binaries
  have been added to Profile. The binaries are able to be used
  by various visualisation tools.
  

• 2cb37a7e
  by James Ennis at 2019-01-18T09:31:24Z

  Merge branch 'jennis/profiling_outputs_binaries' into 'master'
  
  Make the profiler output binaries (which can be used to visualise the data)
  
  See merge request BuildStream/buildstream!1082

• 56ec33cc
  by Valentin David at 2019-01-18T11:14:41Z

  .gitlab-ci.yml: Add overnight tests logs to artifacts.
  

• ad2df651
  by Javier Jardón at 2019-01-18T12:01:56Z

  Merge branch 'valentindavid/overnight-tests-logs-artifacts' into 'master'
  
  .gitlab-ci.yml: Add overnight tests logs to artifacts.
  
  See merge request BuildStream/buildstream!1084

• f874295f
  by Tristan Van Berkom at 2019-01-18T15:59:28Z

  bzr source plugin: Use lock files to avoid corrupting the source cache
  
  This patch by itself fixes #868
  

• 42a2fe3c
  by Tristan Van Berkom at 2019-01-18T15:59:28Z

  bzr source plugins: Remove all of the atomic backup directory business logic
  
  Follow up of last commit which uses exclusive locking to
  protect bzr operations instead.
  

• a8713ed2
  by Tristan Van Berkom at 2019-01-18T15:59:28Z

  testutils/runcli.py: Added cli.get_element_states()
  
  With get_element_state(), you need to invoke BuildStream once
  for every element state you want to observe in a pipeline.
  
  The new get_element_states() reports a dictionary with
  the element state hashed by element name and is better to use
  if you have more than one element to observe the state of.
  

• a895cb2a
  by Tristan Van Berkom at 2019-01-18T15:59:28Z

  tests/frontend/track.py: Use test_track_recurse() to stress test Sources
  
  This causes multiple source instances to interact with the same
  backing data store at the same time, increasing the likelyhood
  of triggering issues around concurrent access.
  
  This more reliably triggers issue #868
  

• 4236bcc7
  by Tristan Van Berkom at 2019-01-18T16:35:23Z

  Merge branch 'tristan/fix-bzr-race' into 'master'
  
  Fix bzr race conditions
  
  Closes #868
  
  See merge request BuildStream/buildstream!1083

• 951a8df1
  by Angelos Evripiotis at 2019-01-18T18:02:22Z

  cli.py: add a hint about '--' to 'bst shell' help

• 9911023f
  by Angelos Evripiotis at 2019-01-18T18:56:21Z

  Merge branch 'aevri/shell_separator_hint' into 'master'
  
  cli.py: add a hint about '--' to 'bst shell' help
  
  See merge request BuildStream/buildstream!1078

• c9ce89d2
  by Tristan Van Berkom at 2019-01-18T19:36:26Z

  _cas/cascache.py: Cleanup directories when removing refs
  
  With out this, empty directories in the refs/heads directory just
  grow unconditionally.
  

• c536ab6a
  by Tristan Van Berkom at 2019-01-18T19:36:26Z

  tests/artifactcache/expiry.py: Test refs directory cleanup
  
  Enhance the test which checks removal of the extract directories
  with an additional check that the ref directories are cleaned up
  when removing artifacts.
  

• 99699ffc
  by Tristan Van Berkom at 2019-01-18T19:36:26Z

  utils.py: Added _tempnamedfile()
  
  When used in a child process in BuildStream, this should be
  used instead of tempfile.NamedTemporaryFile() directly, otherwise
  we fail to cleanup the file on SIGTERM.
  

• 8ce483d4
  by Tristan Van Berkom at 2019-01-18T19:36:26Z

  _cas/cascache.py: Use utils._tempdir() and utils._tempnamedfile()
  
  The direct usage of tempfile.TemporaryDirectory() and
  tempfile.NamedTemporaryFile() here causes leakage of any temporary
  data when the process operating on temporary data is terminated
  with SIGTERM.
  
  Using the utilities ensures that trash is not left behind in
  ~/.cache/buildstream/artifacts/tmp when the user terminates
  BuildStream with ^C.
  

• 73c7252d
  by Tristan Van Berkom at 2019-01-18T20:57:42Z

  Merge branch 'tristan/cas-cleanup-improve' into 'master'
  
  CASCache cleanup improvements
  
  See merge request BuildStream/buildstream!1087

• adfb9291
  by Tristan Van Berkom at 2019-01-18T21:23:05Z

  tests/testutils/python_repo.py: Use subprocess to run sdist
  
  The current approach using setuptools.sandbox.run_setup() was
  causing a spurious (but highly frequent) failure where setuptools
  gets mixed up with it's manipulation of sys.modules and hits a
  RuntimeError as a dictionary changes size while being iterated over.
  
  For instance: https://gitlab.com/BuildStream/buildstream/-/jobs/147967307
  
  Since this already happens in an isolated virtual environment created
  by tox, we should not need additional sandboxing here from setuptools,
  and launching this as a subprocess will be safer.
  

• d114a6bd
  by Tristan Van Berkom at 2019-01-18T22:07:22Z

  Merge branch 'tristan/fix-pip-source-test' into 'master'
  
  tests/testutils/python_repo.py: Use subprocess to run sdist
  
  See merge request BuildStream/buildstream!1090

• bf591ade
  by Chandan Singh at 2019-01-18T22:09:04Z

  Derive import plugin from Element instead of BuildElement
  
  The `import` element is not really a build element. The main purpose of
  the `BuildElement` class is to allow users to run `build-commands`,
  `install-commands` etc. But, `import` does not run such commands.
  Moreover, we already override all the methods provided by
  `BuildElement`. So it only makes it confusing to have it derived from
  `BuildElement` class when it is not a build element.
  
  So, derive it from the base `Element` class instead.
  

• 2233a532
  by Chandan Singh at 2019-01-18T22:46:41Z

  Merge branch 'chandan/import-is-not-buildelement' into 'master'
  
  Derive import plugin from Element instead of BuildElement
  
  See merge request BuildStream/buildstream!1089

• aa3411f9
  by Tristan Maat at 2019-01-18T22:47:46Z

  testutils/runcli.py: Allow removing artifacts from arbitrary dirs
  
  `remove_artifact_from_cache` used a hard-coded path to remove
  artifacts, which wasn't sufficient for integration tests.
  

• fff882fe
  by Tristan Maat at 2019-01-18T22:47:46Z

  widget.py: Avoid "showing 0 lines" messages when there are no lines
  
  This happened when bst is invoked with --message-lines 0 or
  --error-lines 0, and was arguably a little too verbose (the user
  explicitly asked us not to show them any lines, after all).
  
  Fixes #779
  

• e230dedb
  by Tristan Van Berkom at 2019-01-18T23:23:35Z

  Merge branch 'tlater/message-lines' into 'master'
  
  Avoid "showing 0 lines" messages when we're asked to show no lines
  
  Closes #779
  
  See merge request BuildStream/buildstream!1031

• 43797617
  by Richard Maw at 2019-01-21T10:41:37Z

  _gitsourcebase.py: Fetch with explicit refspecs
  
  Old versions of git lack --force and --tags,
  but the same effect can be had by specifying refspecs.
  

• ce8dab0f
  by Richard Maw at 2019-01-21T10:41:37Z

  tests/frontend/workspace.py: Skip test_open_multi_unwritable when root
  
  The test assumes that a directory with write permission removed isn't writable,
  this isn't the case if the process running the tests has CAP_DAC_OVERRIDE
  which is common when running as root.
  

• 50165081
  by Richard Maw at 2019-01-21T10:41:37Z

  tests/sources/git.py: Skip tests that assume too new a git
  
  test_track_invalid_submodule depends on being able to remove a submodule
  by `git rm $submoduledir`, but old versions of git don't update .gitmodules
  so BuildStream still thinks there's a submodule present.
  
  For expediency the test is skipped rather than changed to manually remove
  the entry from .gitmodules if git hasn't done it,
  since in the common case git is new enough to do that itself.
  
  test_git_describe expects --first-parent to find another tag,
  but `bst track` will gracefully degrade if the option doesn't work
  so a different history will be retained with old versions of git.
  
  It's of marginal benefit to add additional cruft
  to test for different output on old versions of git that won't persist forever.
  

• 8677a256
  by Benjamin Schubert at 2019-01-21T10:41:37Z

  Add a Centos runner
  
  Centos is apparently different enough from fedora when running tests.
  

• 33782865
  by Valentin David at 2019-01-21T11:47:18Z

  Merge branch 'richardmaw/centos-oldgit-test-fixes' into 'master'
  
  Fix CentOS
  
  Closes #833
  
  See merge request BuildStream/buildstream!1085

• c00b3782
  by Chandan Singh at 2019-01-21T19:37:53Z

  CONTRIBUTING.rst: Fix formatting of link to pip docs
  
  The link explaining python's requirements files was formatted
  incorrectly (missing underscore at the end). This resulted in the text
  being rendered literally, as opposed to being converted into a
  hyperlink.
  

• 638ceb8a
  by Chandan Singh at 2019-01-21T20:16:38Z

  Merge branch 'chandan/fix-contrib-link' into 'master'
  
  CONTRIBUTING.rst: Fix formatting of link to pip docs
  
  See merge request BuildStream/buildstream!1094

• 383142e3
  by James Ennis at 2019-01-22T12:32:43Z

  using_commands.rst: Split out top level commands and subcommand groups
  

• ac135333
  by James Ennis at 2019-01-22T12:32:43Z

  using_commands.rst: Add the source checkout command
  

• a3fc350f
  by James Ennis at 2019-01-22T12:32:43Z

  Move push and pull to the new artifact subcommand group
  
  This commit also ensures that if we try to use the 'old' commands,
  BuildStream will fail and instruct the user to use the new command.
  

• 9eefe863
  by James Ennis at 2019-01-22T12:32:43Z

  cli: Add artifact checkout subcommand
  
  'artifact checkout' has slightly different behaviour from 'checkout',
  that is, either '--directory' or '--tar' are now required options.
  This is a step towards allowing checkout to take multiple args.
  

• fbd15939
  by James Ennis at 2019-01-22T12:32:43Z

  Mark 'old' checkout command as obsolete
  
  This commit marks 'bst checkout' as a 'hidden' command. If used,
  the user will be prompted to use the new 'bst artifact checkout'
  command.
  
  All tests which used 'bst checkout' have been modified to use
  the new artifact sub-command.
  
  This partially solves #822.
  

• 9e8034e9
  by James Ennis at 2019-01-22T12:32:43Z

  Add NEWS entry describing the deprecation of checkout, pull and push
  

• 1bccf1d2
  by James Ennis at 2019-01-22T12:38:40Z

  using_commands.rst: Split out the artifact subcommands
  

• d9072371
  by James Ennis at 2019-01-22T12:38:40Z

  using_commands.rst: Add artifact log subcommand
  

• 077d5a96
  by James Ennis at 2019-01-22T12:41:29Z

  Change bst checkout/pull/push references to bst artifact checkout/pull/push
  

• ea2fbe4d
  by James Ennis at 2019-01-22T13:02:14Z

  man: Regenerate all of our man pages
  
  Due to an upstream click_man issue:
  https://github.com/click-contrib/click-man/issues/10
  generating the man pages with our current setup.py only generates
  a man page for bst-artifact-server, our first entry point.
  I then had to remove this entry point from setup.py and regenerate to
  obtain man pages for the commands in cli.py
  

• 9c2a6b93
  by James Ennis at 2019-01-22T15:08:17Z

  Merge branch 'jennis/migrate_pull_push_commands' into 'master'
  
  Move push/pull/checkout to the artifact subcommand group
  
  See merge request BuildStream/buildstream!1045

• 35377522
  by James Ennis at 2019-01-22T15:28:13Z

  cli.py: Add an obsoletion note to our deprecated commands.
  
  Click 7.0 allows us to declare commands as 'hidden'.
  However, sphinx-click and click-man still generate these hidden
  commands in the documentation and man pages, respectively.
  
  This is a stop gap solution until the upstream issues (tracked in
  issues #879 and #881) have been addressed.
  

• 76148785
  by James Ennis at 2019-01-22T16:15:20Z

  Merge branch 'jennis/add_obsoletion_note' into 'master'
  
  Add an obsoletion note to our deprecated commands.
  
  See merge request BuildStream/buildstream!1097

• 1443c542
  by Tristan Van Berkom at 2019-01-22T16:26:56Z

  _scheduler: Fix dont display a failure for terminated jobs
  
  This fixes a recent regression introduced in c2fc2a5ea
  

• 6e5c9987
  by Tristan Van Berkom at 2019-01-22T17:14:50Z

  Merge branch 'tristan/fix-terminated-status' into 'master'
  
  _scheduler: Fix dont display a failure for terminated jobs
  
  See merge request BuildStream/buildstream!1096

• 3315b9a1
  by Tristan Van Berkom at 2019-01-22T18:46:24Z

  tests/integration/pullbuildtrees.py: Fix the non-integration case.
  
  This test has one test case which is marked as an integration test,
  and the other is not an integration test, but was using the integration
  cli. The integration cli does not work correctly if not run in
  integration mode.
  
  This was causing an error locally in conftest.py when trying to
  create a tmpdir inside a nonexisting integration cache directory.
  

• 56e857f4
  by Tristan Van Berkom at 2019-01-22T19:59:57Z

  Merge branch 'tristan/fix-pullbuildtrees-test' into 'master'
  
  tests/integration/pullbuildtrees.py: Fix the non-integration case.
  
  See merge request BuildStream/buildstream!1098

• 3895571a
  by Chandan Singh at 2019-01-23T00:45:47Z

  tox.ini: Unpin sphinx dependency
  
  https://github.com/rtfd/sphinx_rtd_theme/pull/672 has been fixed
  upstream, and the newer versions of `sphinx_rtd_theme` do not break
  search functionality with Sphinx >= 1.8.
  

• 528e8ed7
  by Chandan Singh at 2019-01-23T01:31:49Z

  Merge branch 'chandan/unpin-sphinx' into 'master'
  
  tox.ini: Unpin sphinx dependency
  
  See merge request BuildStream/buildstream!1093

• 0098a900
  by Tristan Van Berkom at 2019-01-23T13:57:53Z

  tests: Migrated cache quota test into artifactcache/cache_size.py
  
  Instead of sitting mysteriously alone in internals/utils.py
  

• 9be31641
  by Tristan Van Berkom at 2019-01-23T13:57:53Z

  _artifactcache.py: Raise ArtifactError() when quota size exceeds disk space.
  
  This is not an error related to loading data, like a parse error
  in the quota specification is, but a problem raised by the artifact
  cache - this allows us to assert more specific machine readable
  errors in test cases (instead of checking the string in stderr, which
  this patch also fixes).
  
  This also removes a typo from the error message in the said error.
  
    * tests/artifactcache/cache_size.py
  
      Updated test case to expect the artifact error, which consequently
      changes the test case to properly assert a machine readable error
      instead of asserting text in the stderr (which is the real, secret
      motivation behind this patch).
  
    * tests/artifactcache/expiry.py: Reworked test_invalid_cache_quota()
  
      Now expect the artifact error for the tests which check configurations
      which create caches too large to fit on the disk.
  

• 3a6e953f
  by Tristan Van Berkom at 2019-01-24T01:59:13Z

  Merge branch 'tristan/insufficient-storage-error' into 'master'
  
  Tristan/insufficient storage error
  
  See merge request BuildStream/buildstream!1102

• 68339b19
  by Tristan Van Berkom at 2019-01-24T05:01:55Z

  tests/testutils/runcli.py: Make get_element_states() take a list of targets
  
  Instead of a single target, we can always provide a single target
  in a list.
  

• 6ecc2b0a
  by Tristan Van Berkom at 2019-01-24T05:01:55Z

  tests/artifactcache/expiry.py: Refactored to use get_element_states()
  

• 1140aed5
  by Tristan Van Berkom at 2019-01-24T05:01:55Z

  tests/elements/filter.py: Refactored to use get_element_states()
  

• b7ea8b74
  by Tristan Van Berkom at 2019-01-24T05:01:55Z

  tests/sources/remote.py: Refactored to use get_element_states()
  

• c2c004f9
  by Tristan Van Berkom at 2019-01-24T05:01:55Z

  tests/frontend/pull.py: Refactored to use get_element_states()
  

• 21b2958b
  by Tristan Van Berkom at 2019-01-24T05:01:55Z

  tests/frontend/push.py: Refactored to use get_element_states()
  

• 83fcaa9f
  by Tristan Van Berkom at 2019-01-24T05:01:56Z

  tests/frontend/track.py: Refactored to use get_element_states()
  

• 46eb3018
  by Tristan Van Berkom at 2019-01-24T05:01:56Z

  tests/frontend/workspace.py: Refactored to use get_element_states()
  

• 341b131b
  by Tristan Van Berkom at 2019-01-24T06:13:57Z

  Merge branch 'tristan/test-element-states' into 'master'
  
  Reduce number of calls to `bst show` in tests
  
  See merge request BuildStream/buildstream!1103

• ef2b4648
  by Tristan Van Berkom at 2019-01-24T06:14:20Z

  tests/frontend/track.py: test_track_error_cannot_write_file() fixup
  
  This tests how BuildStream reacts when it fails to write the tracking
  results to the element files or project.refs file, which is an operation
  that plugins do not play a part in.
  
  As such, removing the per repo kind parameterization from this test
  as multiple runs are redundant here.
  

• 40c18174
  by Tristan Van Berkom at 2019-01-24T07:00:50Z

  Merge branch 'tristan/track-test-reduce' into 'master'
  
  test_track_error_cannot_write_file() fixup
  
  See merge request BuildStream/buildstream!1104

• 5b0bba85
  by Jonathan Maw at 2019-01-24T10:15:06Z

  Test that tracking in workspaces actually works
  
  Previously, it merely tested that buildstream did not fall other,
  rather than whether it did anything useful.
  

• 58d7d722
  by Jonathan Maw at 2019-01-24T10:18:01Z

  tests: Test that fetching an open workspace will fetch its dependencies
  
  Previously, there was no way of detecting whether fetching happened, as
  an element with an open workspace will not be fetched.
  

• 24bd8994
  by Jürg Billeter at 2019-01-24T13:36:33Z

  Merge branch 'jonathan/test-missing-workspace-guessing' into 'master'
  
  Add tests to cover reinstated support for guessing targets
  
  See merge request BuildStream/buildstream!1042

• e03dd5fc
  by Jürg Billeter at 2019-01-24T13:37:28Z

  _frontend/cli.py: Guess element also for bst build --all
  
  There is no reason to disallow guess_element() for bst build --all.
  

• 3850274b
  by Phillip Smyth at 2019-01-24T13:37:28Z

  projectconfig.yaml: Add key for default targets
  
  _versions.py: Bump format version
  

• ab72d384
  by Jürg Billeter at 2019-01-24T13:37:28Z

  _project.py: Add get_default_target() and get_default_targets() methods
  
  _frontend/cli.py: Use new methods.
  
  Based on patches by Phillip Smyth.
  

• ee2d8434
  by Jürg Billeter at 2019-01-24T13:37:28Z

  _stream.py: Add ignore_junction_targets parameter
  
  This filters out junctions from the list of targets.
  

• 2f175f0a
  by Jürg Billeter at 2019-01-24T13:37:28Z

  _frontend/cli.py: Ignore junctions in default targets where appropriate
  
  Junctions cannot be built, pulled, or pushed. Specifying a junction on
  the command line for these commands will result in an error. However,
  junctions may be in the list of default targets, so they need to be
  ignored for build, pull, and push commands.
  

• 5a351dee
  by Jürg Billeter at 2019-01-24T13:38:50Z

  format_project.rst: Add documentation for default targets
  

• 77414518
  by Phillip Smyth at 2019-01-24T13:38:50Z

  tests/frontend: Add default target tests for bst show and build
  

• ce1c10ec
  by Jürg Billeter at 2019-01-24T13:38:50Z

  tests/frontend/fetch.py: Add default target test for bst source fetch
  

• 99b5c0af
  by Jürg Billeter at 2019-01-24T13:38:50Z

  tests/frontend/pull.py: Add default target test for bst push/pull
  

• 3642c8e7
  by Jürg Billeter at 2019-01-24T13:38:50Z

  tests/frontend/buildcheckout.py: Add default target test with junction
  
  Test that `bst build` does not fail in a project where the list of
  default targets includes a junction (junctions cannot be built).
  

• 50c5159f
  by Phillip Smyth at 2019-01-24T13:38:50Z

  NEWS: Add entry for default target feature
  

• 05587f22
  by Jürg Billeter at 2019-01-24T15:10:08Z

  Merge branch 'issue-638-validate-all-files' into 'master'
  
  Add support for default targets
  
  See merge request BuildStream/buildstream!925

• 3e36e363
  by Tristan Van Berkom at 2019-01-24T16:55:24Z

  _scheduler/resources.py: Dont error out in unregister_exclusive_interest()
  
  Don't require the interest to be registered, just discard any interest,
  this function just sets a bit in a mask, and is not intended to maintain
  a balance like the reserve() function is.
  

• ce01f87e
  by Tristan Van Berkom at 2019-01-24T16:55:24Z

  _scheduler/scheduler.py: Run cache size exclusively at startup
  
  When running any session that has Queues which require Resource.CACHE,
  check if our loaded estimated size exceeds the quota, and if so;
  lock the Resource.CACHE resource exclusively right away and run
  an exclusive initial cache size job.
  
  This ensures we cleanup first before doing anything which might
  add to the cache at startup time, if deemed needed.
  
  This is a partial fix for issue #737
  

• 2479e8df
  by Tristan Van Berkom at 2019-01-24T16:55:24Z

  _frontend/widget.py: Render core messages more like other messages
  
  In order to test when core activities occur by parsing the stderr
  in tests, we should make the messages conform more.
  
  At the same time, this restores alignment of columns in core
  messages with the element processing related messages.
  
  Also, _scheduler/scheduler.py is updated to make it's activity names
  conform to the (current) 5 character limit for the sake of alignment.
  
  The tests/frontend/logging.py test gets it's regexes updated for
  the log lines it checks for in stderr.
  

• fdb8ff65
  by Tristan Van Berkom at 2019-01-24T16:55:24Z

  tests/artifactcache/expiry.py: Test that expiry happens first
  

• acd0bf22
  by Tristan Van Berkom at 2019-01-24T18:01:41Z

  Merge branch 'tristan/cache-management' into 'master'
  
  Cache management fixes
  
  See merge request BuildStream/buildstream!1091

• 41f03296
  by Tristan Van Berkom at 2019-01-24T18:02:08Z

  _frontend/app.py: Initialize logging before preflighting the artifact cache
  
  The artifact cache emits messages, and we want to allow that in preflight.
  

• 24ca2f46
  by Tristan Van Berkom at 2019-01-24T18:02:08Z

  utils.py: Add _get_volume_size()
  
  We can streamline this call to os.statvfs() in a few places.
  

• 9fd9fbb9
  by Tristan Van Berkom at 2019-01-24T18:02:08Z

  _artifactcache.py: Refactored to use utils._get_volume_size()
  
  This will benefit from a better UtilError being raised, and
  and turns the artifact cache's local function into a one liner.
  
  The loop which finds the first existing directory in the
  given path has been removed, being meaningless due to the
  call to os.makedirs() in ArtifactCache.__init__().
  
  The local function was renamed to _get_cache_volume_size() and
  no longer takes any arguments, which is more suitable for the
  function as it serves as a testing override surface for
  unittest.mock().
  
  The following test cases which use the function to override
  the ArtifactCache behavior have been updated to use the new
  overridable function name:
  
     tests/artifactcache/cache_size.py
     tests/artifactcache/expiry.py
  

• 7ee0c579
  by Tristan Van Berkom at 2019-01-24T18:02:08Z

  _artifactcache.py: Added ArtifactCacheUsage()
  
  A simple object which creates a snapshot of current
  usage statistics for easy reporting in the frontend.
  

• 353293b6
  by Tristan Van Berkom at 2019-01-24T18:02:08Z

  _context.py: Added get_artifact_cache_usage()
  
  A frontend facing API for obtaining usage statistics.
  
  I would have put this on Stream instead, but the Context
  seems to be the de facto place for looking up the artifact cache
  in general so let's put it here.
  

• 51ed36de
  by Tristan Van Berkom at 2019-01-24T18:02:08Z

  _frontend/widget.py: Added cache usage entry in the startup heading
  

• 5797238b
  by Tristan Van Berkom at 2019-01-24T18:02:08Z

  _frontend/status.py: Added Cache size usage indicator to status bar
  
  This also adds some comments around the main status bar heading
  rendering function.
  

• 8074ebf4
  by Tristan Van Berkom at 2019-01-24T18:02:08Z

  _artifactcache.py: Add status messages in cache management operations
  
  Added some useful status messages when:
  
    * Calculating a new artifact cache usage size
  
    * Starting a cleanup
  
    * Finishing a cleanup
  
  Also enhanced messaging about what was cleaned up so far when
  aborting a cleanup.
  

• 49c11bc8
  by Tristan Van Berkom at 2019-01-24T18:02:08Z

  _artifactcache.py: Added client progress callback to ArtifactCache.clean()
  

• 3616e939
  by Tristan Van Berkom at 2019-01-24T18:02:08Z

  _scheduler/jobs/job.py: Allow subclasses to message the frontend
  

• bcd19266
  by Tristan Van Berkom at 2019-01-24T18:02:08Z

  _scheduler/jobs/cleanupjob.py: Update cache size while processing
  
  Updates the known cache size in the main process while the cleanup
  process is ongoing, so that the status indicators update live
  while the cleanup happens.
  

• 9c33107f
  by Tristan Van Berkom at 2019-01-24T18:02:08Z

  _artifactcache.py: Correcting API documenting comment for remove()
  
  This seems to have been copy/pasted from cascache, and
  documents the function to possibly return None if defer_prune
  was specified, but this function does not expose defer_prune.
  

• a2140d74
  by Tristan Van Berkom at 2019-01-24T19:44:52Z

  Merge branch 'tristan/cache-management-logging' into 'master'
  
  Cache management logging enhancements
  
  See merge request BuildStream/buildstream!1105

• d5847462
  by James Ennis at 2019-01-25T16:35:21Z

  _profile.py: Added a new profiling topic, scheduler
  
  profile_start() and profile_end() calls have been incorporated into
  Scheduler.run()
  

• 56a3954c
  by James Ennis at 2019-01-25T16:35:21Z

  _profile.py: Added a new profiling topic, load-selection
  
  profile_start() and profile_end() calls have been added to
  Stream.load_selection()
  

• 116da6d7
  by James Ennis at 2019-01-25T16:35:21Z

  _profile.py: Update copyright statement
  

• 137d31cd
  by James Ennis at 2019-01-25T17:59:22Z

  Merge branch 'jennis/add_new_profile_topic' into 'master'
  
  Add new 'scheduler' and 'load-selection' profiling topics
  
  See merge request BuildStream/buildstream!1088

• 22b3a0c1
  by Tristan Van Berkom at 2019-01-25T18:35:36Z

  _artifactcache.py: Don't require the quota to be available on disk.
  
  Instead only rely on the headroom to be enough to protect against
  out of space conditions. The headroom can become configurable as
  a separate step is required.
  
  The changes to achieve this are:
  
    * Rename ArtifactCache.has_quota_exceeded() to ArtifactCache.full().
  
    * ArtifactCache.full() now also reports True if the available
      space on the artifact cache volume is smaller than the headroom.
  
      This ensures jobs get triggered to cleanup the cache when
      reaching the end of the disk.
  
    * When loading the artifact quota, it is now only an error if
      the quota exceeds the overall disk space, not if it does not
      fit in the available space.
  
      It is still a warning if the quota does not fit in the
      available space on the artifact cache volume.
  
    * Updated scheduler.py and buildqueue.py for the API rename
  
    * tests: Updated the artifactcache/expiry.py test for its
      expectations in this regard.
  
      Added a new test to test an error when quota was specified to
      exceed total disk space, and adjusted the existing tests to
      expect a warning when the quota does not fit in the available
      space.
  
  This fixes issue #733 and #869.
  

• 27ca6cc7
  by Tristan Van Berkom at 2019-01-25T21:18:15Z

  Merge branch 'tristan/cache-quota-max-only' into 'master'
  
  _artifactcache.py: Don't require the quota to be available on disk.
  
  Closes #869 and #733
  
  See merge request BuildStream/buildstream!1106

• 89ba2abe
  by Chandan Singh at 2019-01-25T23:50:49Z

  tox.ini: Add environment to update man pages
  
  Previously, one had to manually install `click-man` package, and
  remember to run the correct command. Now, we can simply run `tox -e man`
  to update the man pages.
  

• d7438688
  by Chandan Singh at 2019-01-25T23:50:49Z

  man: Refresh man pages
  
  Notable changes:
  
  * New `source` and `artifact` command groups
  * Man pages corresponding to obsolete commands, that were marked as
    hidden in Click, have now been removed.
  
  Fixes #881.
  

• 4edbbd27
  by Chandan Singh at 2019-01-25T23:50:49Z

  setup.py, CONTRIBUTING.rst: Recommend using tox to generate man pages
  
  Simplify our docs, by requesting users to run `tox -e man` to update man
  pages, instead of manually installing `click-man` and running the
  command manually.
  

• a3e7aee8
  by Chandan Singh at 2019-01-25T23:50:49Z

  setup.py: Do not error out when man directory is empty/missing
  
  If the `man` directory is empty, then it won't be copied in the source
  distribution, and `list_man_pages()` will throw an exception when trying
  to list files in a non-existent directory. This prevents us from
  installing the BuildStream package when the man pages are not there.
  
  The most common use-case for this is when we want to re-generate the man
  pages but want to install the package before re-generating them.
  

• 222753ac
  by Tristan Van Berkom at 2019-01-26T05:10:36Z

  Merge branch 'chandan/toxic-man' into 'master'
  
  Generate man pages using tox & update them
  
  Closes #880 and #881
  
  See merge request BuildStream/buildstream!1107

• 5df4105a
  by Angelos Evripiotis at 2019-01-28T10:13:40Z

  news: fix 'osbolete' spelling
  

• 9c981eff
  by Angelos Evripiotis at 2019-01-28T10:17:57Z

  Fixup refs to 'bst track'
  
  Now that 'bst track' is obsolete, change guidance to refer to the
  replacement 'bst source track' instead.
  

• bef80291
  by Angelos Evripiotis at 2019-01-28T10:19:17Z

  Fixup refs to 'bst fetch'
  
  Now that 'bst fetch' is obsolete, change guidance to refer to the
  replacement 'bst source fetch' instead.
  

• 564cb245
  by Angelos Evripiotis at 2019-01-28T11:36:16Z

  Merge branch 'aevri/bst_track_guidance' into 'master'
  
  Fixup refs to 'bst track' and 'bst fetch'
  
  See merge request BuildStream/buildstream!1086

• a3e2cdd2
  by Tom Pollard at 2019-01-28T12:14:40Z

  _stream.py: Add use_artifact_config opt arg for load_selection()
  
  use_artifact_config added as an optional default arg, allowing
  for loading of given elements artifact remote config.
  

• 805baf7d
  by Tom Pollard at 2019-01-28T12:14:40Z

  Download buildtrees on demand for bst shell --use-buildtree
  
  Provide bst shell --use-buildtree the ability to attempt to
  acquire missing buildtrees, given respective option, user
  pull-buildtree context and remote availability.
  
  _frontend/cli.py: Refactor logic for determining --use-buildtree
  option with given opportunity to attempt pulling a non-local
  buildtree. Element loaded with artifact_config to allow remote
  querying.
  
  _stream.py: With given user option and element state, construct
  PullQueue to fetch remote buildtree. Continue or Error without
  buildtree if cannot be attained.
  
  tests/integration/build-tree.py: Update to support new usecases
  

• 38356932
  by Tom Pollard at 2019-01-28T13:44:37Z

  Merge branch 'tpollard/829' into 'master'
  
  Download buildtrees on demand for bst shell --use-buildtree
  
  Closes #829
  
  See merge request BuildStream/buildstream!1050

• 3590ca8c
  by Abderrahim Kitouni at 2019-01-28T14:59:19Z

  requirements/requirements.in: require protobuf >= 3.6
  
  This is needed since 0f2bc3754
  

• 39b952dc
  by Abderrahim Kitouni at 2019-01-28T15:19:04Z

  requirements/requirements.in: require Click >= 7.0
  
  This is needed since 629a6e524, and was lost in the conversion to requirements.in
  

• 80b36d0c
  by Javier Jardón at 2019-01-28T17:54:52Z

  Merge branch 'abderrahim/protobuf-version' into 'master'
  
  requirements/requirements.in: update minimum versions
  
  Closes #884
  
  See merge request BuildStream/buildstream!1114

• a1ab48da
  by Valentin David at 2019-01-28T21:30:26Z

  Fix crash when spawned job completes very fast
  
  Job can complete before we return from `Job.span()` to
  `Scheduler._spawn_job()`, so that `_active_jobs` would not yet contain
  the job.
  
  This would print a stack on the console and try to run a second time
  the job which can have unexpected effects.
  
  In order to reproduce the issue, in
  `buildstream/_scheduler/jobs/job.py`, in `Job.spawn`,
  add a call to `time.sleep()` right before call to
  `asyncio.get_child_watcher()`.
  
  This fixes issue #857.
  

• 2fcb4491
  by Jürg Billeter at 2019-01-28T22:36:22Z

  Merge branch 'valentindavid/crash_in_scheduler_857' into 'master'
  
  Fix crash when spawned job completes very fast
  
  Closes #857
  
  See merge request BuildStream/buildstream!1095

• 1c05a092
  by Valentin David at 2019-01-29T05:58:17Z

  Fix type of error codes in CAS server
  
  Fixes #882.
  

• 785da59c
  by Jürg Billeter at 2019-01-29T06:49:10Z

  Merge branch 'valentindavid/wrong_type_in_status_code' into 'master'
  
  Fix type of error codes in CAS server
  
  Closes #882
  
  See merge request BuildStream/buildstream!1099

• ddef91ea
  by Valentin David at 2019-01-29T07:23:35Z

  Make sure testing cache directory exists
  
  Fixes #873
  

• 6a4c8611
  by Jürg Billeter at 2019-01-29T08:18:45Z

  Merge branch 'valentindavid/make_cache_dir' into 'master'
  
  Make sure testing cache directory exists
  
  Closes #873
  
  See merge request BuildStream/buildstream!1092

• 86c8e414
  by Angelos Evripiotis at 2019-01-29T10:39:29Z

  BREAK:remove unconditional 'are you sure?' prompts
  
  This is a breaking change, as it affects behaviour that people might be
  relying on. An entry has been added to NEWS.
  
  As proposed on the mailing list, this change removes the unconditional
  prompts on:
  
      o: bst workspace reset
      o: bst workspace close --remove-dir
  
  If interactive, these commands would always interrupt you with a prompt
  like this:
  
      This will remove all your changes, are you sure?
  
  This seems like it may just save someone's work some time. It may also
  condition folks to hit 'y' quickly without thinking.
  
  This change also makes the non-interactive behaviour consistent with the
  interactive behaviour in the default case. There is also the case of the
  prompt configured by 'really-workspace-close-project-inaccessible',
  which may be tackled in later work.
  
  This change also removes the new config options to suppress those
  prompts, and their associated news entry.
  
  The relevant bit of the mailing list conversation is here:
  https://mail.gnome.org/archives/buildstream-list/2018-December/msg00106.html
  
  The issue to make interactive and non-interactive behaviour consistent
  is here:
  https://gitlab.com/BuildStream/buildstream/issues/744
  

• aae35e13
  by Angelos Evripiotis at 2019-01-29T12:08:07Z

  Merge branch 'aevri/are_you_sure' into 'master'
  
  BREAK:remove unconditional 'are you sure?' prompts
  
  See merge request BuildStream/buildstream!1061

• 03111d39
  by Dor Askayo at 2019-01-30T10:35:06Z

  filter.py: don't recurse when staging dependencies
  
  Also bump the element's version so cached artifacts would be
  invalidated.
  
  Fixes #883
  

• 7256bb0c
  by James Ennis at 2019-01-30T11:34:04Z

  Merge branch 'doraskayo/filter-indirect-deps-fix' into 'master'
  
  filter.py: don't recurse when staging dependencies
  
  Closes #883
  
  See merge request BuildStream/buildstream!1110

• f3c1d7b8
  by Jim MacArthur at 2019-01-31T10:22:33Z

  _sandboxremote.py: Create marked directories before running
  
  The other two sandboxes (BWrap and chroot) both do this because
  marked directories may be mount points. _cache_artifact requires
  install-root to be present, and existing BuildStream projects
  may rely on these too.
  

25 changed files:

• .gitlab-ci.yml
• CONTRIBUTING.rst
• NEWS
• buildstream/_artifactcache/artifactcache.py → buildstream/_artifactcache.py
• buildstream/_artifactcache/__init__.py → buildstream/_cas/__init__.py
• buildstream/_artifactcache/cascache.py → buildstream/_cas/cascache.py
• + buildstream/_cas/casremote.py
• buildstream/_artifactcache/casserver.py → buildstream/_cas/casserver.py
• buildstream/_context.py
• buildstream/_exceptions.py
• buildstream/_frontend/app.py
• buildstream/_frontend/cli.py
• buildstream/_frontend/status.py
• buildstream/_frontend/widget.py
• buildstream/_gitsourcebase.py
• buildstream/_loader/loader.py
• buildstream/_pipeline.py
• buildstream/_profile.py
• buildstream/_project.py
• buildstream/_scheduler/jobs/cachesizejob.py
• buildstream/_scheduler/jobs/cleanupjob.py
• buildstream/_scheduler/jobs/job.py
• buildstream/_scheduler/queues/buildqueue.py
• buildstream/_scheduler/queues/queue.py
• buildstream/_scheduler/resources.py

The diff was not included because it is too large.

—
View it on GitLab.
You're receiving this email because of your account on gitlab.com. If you'd like to receive fewer emails, you can adjust your notification settings.