[Notes] [Git][BuildStream/buildstream][bschubert/bst-checkout-build] 164 commits: README: use correct links for documentation website

[Benjamin Schubert <gitlab mg gitlab com>]

Title: GitLab

Benjamin Schubert pushed to branch bschubert/bst-checkout-build at BuildStream / buildstream

Commits:

• edc07536
  by Tiago Gomes at 2018-10-17T13:49:15Z

  README: use correct links for documentation website
  

• 155296a7
  by Tiago Gomes at 2018-10-17T13:49:15Z

  setup.py: use correct link for documentation website
  

• 131f95eb
  by Tiago Gomes at 2018-10-17T14:15:38Z

  Merge branch 'tiagogones/update-links-to-documentation' into 'master'
  
  Update links to documentation website
  
  See merge request BuildStream/buildstream!876

• c327d1f3
  by Valentin David at 2018-10-17T14:33:40Z

  Change URL to the Alpine tarball
  

• eeb334f2
  by Javier Jardón at 2018-10-17T15:03:17Z

  Merge branch 'valentindavid/move_alpine_tarball' into 'master'
  
  Change URL to the Alpine tarball
  
  See merge request BuildStream/buildstream!880

• b8f69946
  by Daniel Silverstone at 2018-10-18T09:32:09Z

  _loader/loader.py: Be more explanatory in CircDep exception
  
  When a circular dependency is detected, the reported error hilights
  only one part of the loop in question.  This change ensures that
  the entire loop is listed so that the user has a greater chance of
  easily determining the correct fix.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 892866e1
  by richardmaw-codethink at 2018-10-18T10:04:32Z

  Merge branch 'danielsilverstone-ct/report-loop' into 'master'
  
  _loader/loader.py: Be more explanatory in CircDep exception
  
  See merge request BuildStream/buildstream!878

• f1aa0974
  by Javier Jardón at 2018-10-18T14:13:35Z

  Fix python warnings: Use collections.abc instead collections
  
  Since python 3.3, collections has been moved to collections.abc module.
  For backwards compatibility, they continue to be visible in this module through Python 3.7.
  Subsequently, they will be removed entirely.
  
  See https://docs.python.org/3/library/collections.html
  

• f1550c8e
  by Valentin David at 2018-10-18T16:02:45Z

  Merge branch 'jjardon/warnings_collections' into 'master'
  
  Fix python warnings: Use collections.abc instead collections
  
  See merge request BuildStream/buildstream!883

• 1c24979a
  by Angelos Evripiotis at 2018-10-19T21:17:01Z

  Spelling fixes: 'affect', not 'effect' as verb
  
  Fix most instances of 'effect' being used as a verb instead of a noun.
  Skipped generated instance in doc/source/conf.py.
  

• 43a2eee3
  by Angelos Evripiotis at 2018-10-19T21:48:59Z

  Merge branch 'aevri/affect' into 'master'
  
  Spelling fixes: 'affect', not 'effect' as verb
  
  See merge request BuildStream/buildstream!885

• 12719f0d
  by Jürg Billeter at 2018-10-22T17:05:41Z

  _artifactcache: Rename update_atime() to update_mtime()
  
  os.utime() updates both, however, list_artifacts() sorts refs by mtime,
  i.e., atime is irrelevant.
  

• a7a28d14
  by Jürg Billeter at 2018-10-22T17:05:41Z

  _artifactcache: Fix ref in update_mtime()
  
  get_artifact_fullname() is required to construct the ref. The cache key
  alone does not suffice.
  

• be29e0f5
  by Jürg Billeter at 2018-10-22T17:53:26Z

  Merge branch 'juerg/cas-mtime' into 'master'
  
  _artifactcache: Fix ref in update_mtime()
  
  See merge request BuildStream/buildstream!882

• b74aca1a
  by Jürg Billeter at 2018-10-23T09:22:19Z

  tests/testutils/artifactshare.py: Fix has_artifact() to match core code
  
  Replace path separator with '-' to match Element.normal_name.
  

• c7dda150
  by Jürg Billeter at 2018-10-23T09:48:00Z

  Merge branch 'juerg/test-artifactshare' into 'master'
  
  tests/testutils/artifactshare.py: Fix has_artifact() to match core code
  
  See merge request BuildStream/buildstream!890

• 74c115b9
  by Angelos Evripiotis at 2018-10-23T10:07:31Z

  plugins/sources: refactor, rm needless ExitStack-s
  
  Remove two uses of ExitStack that aren't necessary and make the code
  a little harder to follow.
  

• ecb58b42
  by Phil Dawson at 2018-10-23T10:33:47Z

  Merge branch 'aevri/rm-exitstack' into 'master'
  
  plugins/sources: refactor, rm needless ExitStack-s
  
  See merge request BuildStream/buildstream!888

• aa0cbf5d
  by Martin Blanchard at 2018-10-23T10:54:40Z

  _sandboxremote.py: Try to reopen operation steam on failure
  
  The REAPI allows a client to reconnect to an ongoing operation stream by
  providing a WaitExecution(). If implemented on server side, BuildStream
  will try to recover from connection errors using it.
  
  https://gitlab.com/BuildStream/buildstream/issues/630
  

• 552f5fc6
  by Jim MacArthur at 2018-10-23T11:19:48Z

  Merge branch 'mablanch/630-remote-execution-reconn' into 'master'
  
  Handle connection losses during remote build execution
  
  See merge request BuildStream/buildstream!806

• 276d2195
  by Angelos Evripiotis at 2018-10-25T10:17:49Z

  _downloadablefilesource: handle ValueError-s
  

• 2d012a21
  by Angelos Evripiotis at 2018-10-25T10:17:49Z

  Merge branch 'aevri/unknown-uri-452' into 'master'
  
  _downloadablefilesource: handle ValueError-s
  
  Closes #452
  
  See merge request BuildStream/buildstream!893

• cf4741ce
  by James Ennis at 2018-10-25T13:05:43Z

  docs: Add helpful notes to artifact server configuration instructions
  

• 5737bb06
  by James Ennis at 2018-10-25T13:05:43Z

  using_configuring_artifact_server.rst: Fix incorrect reference to bst push (with link)
  

• 28317622
  by James Ennis at 2018-10-25T13:35:22Z

  Merge branch 'jennis/patch_remote_cache_docs' into 'master'
  
  Improve our documentation for configuring an artifact server
  
  See merge request BuildStream/buildstream!899

• d2743c52
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _context.py: Remove useless `return` from `Context.message`
  
  This pointless bare `return` was causing modern pylint to raise an
  error.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 95adf09e
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _variables.py: Fix lint issue in `find_recursive_variable`
  
  The python `for`/`else` construct only makes sense if the `for` loop body
  has a `break` statement.  Otherwise the use of `else` causes pylint
  to raise an error.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 9359c067
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _workspaces.py: Simplify chained comparison
  
  In `_parse_workspace_config` there was a chained version comparison
  which is simplified in this commit to quieten a pylint issue.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 796bd8c9
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _yaml.py: use `in (a,b)` to simply boolean checks
  
  Where we use a construct `val == foo or val == bar` we can instead use
  `val in (foo, bar)` which pylint prefers.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 5a3a537f
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _yaml.py: Quieten a lint in Python < 3.6
  
  For Python before 3.6, `path.resolve()` could not take the `strict`
  keyword argument.  Linting on such Python versions will raise an unnecessary
  issue given the check present.  As such, quieten that lint.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• b279c730
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _yamlcache.py: Correct bug in `YamlCache._get_filepath()`
  
  The correct file path was computed but never returned.  This was
  never caught because of a test missing consumption of one of its
  parameters.  This commit resolves the bug mentioned in #722 leaving
  the test to be amended by a future commit.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 527f3b32
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _yamlcache.py: Correct braino in variable name
  
  This braino would mean that we'd fail to raise the right exception
  if we failed to find the right project object when unpickling the
  yaml cache.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 6d0c697d
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _yamlcache.py: Remove unused imports
  
  The imports of _cachekey and utils were unused.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 32323695
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  buildelement.py: Quieten a lint about `__commands`
  
  Unfortunately elements don't override `__init__` in general, so
  we have to quieten pylint here.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• c5c0654d
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  element.py: Simplify some conditions with `in (foo, bar)`
  
  Where we have conditions of the form `var == foo or var == bar` it
  can be simplified to `var in (foo, bar)` which pylint prefers.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 4f84e2f6
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  element.py: Silence pylint warning about abstract method
  
  Unfortunately pylint can't understand that we'll always use subclasses
  of Element and as such complains of a no-return function (`assemble()`)
  having its return value used.  This quietens that warning.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• b4645745
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  plugin.py: use dict.get() instead of an if statement
  
  Pylint recommends the use of dict.get() rather than a multi-line if
  statement for handling optional keyword arguments.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 68ec9bcf
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  source.py: Quiet several pylint issues with abstract methods
  
  A number of times, abstract methods are called which pylint cannot
  possibly understand will only ever happen in subclasses which implement
  the abstract methods.  This silences those specific warnings
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 1bf0a031
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  source.py: Remove unused local variable
  
  Remove `source_kind` as it was unused.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• e9c6db1e
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  source.py: Remove unused variable
  
  Remove the `context` variable which was unused.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 5a187a1e
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  utils.py: Disable lint warning about popen
  
  We use `preexec_fn` in our use of Popen.  Since there are threads
  in the codebase, this is not necessarily safe.  However we go to
  great lengths to ensure that the main process doesn't spawn threads
  (they're used in gRPC a lot) and so it should be safe for our use.
  As such, we disable the lint here.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• b368f57f
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  utils.py: Group stat imports
  
  The imports of `stat` and `S_ISDIR` from `stat` should be grouped to
  satisfy pylint.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• b45fea19
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _artifactcache/artifactcache.py: Remove superfluous parens
  
  The pylint report dislikes the superfluous parens which used to
  be here in this assert.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 3e4a6507
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _artifactcache/artifactcache.py: Silence lint issues
  
  The pylint tool is unable to understand that the abstract methods
  in the artifact cache will only be called when there's concrete
  implementations behind them.  Silence the lint errors for these
  specific calls.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 447780b0
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _frontend/app.py: Remove unused import of `resource`
  
  The `resource` import was unused.  Removed.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 247b9fa8
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _frontend/status.py: Reorder imports
  
  The `curses` import is "standard" whereas `click` is not.  As such, despite
  alphabetical order normally being the rule, `curses` must come before `click`.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 522867c8
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _fuse/{hardlinks,mount}.py: Remove dangerous use of {} in defaults
  
  Since default values in arguments to functions and methods are created once
  at the compilation of the code, it is dangerous to include list and dict
  literals in them.  This changes the use of {} to None.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 31a7bce5
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _fuse/mount.py: Deal with abstract method lint issue
  
  Unfortunately pylint can't tell this is an abstract method which will
  only ever end up invoked from a class which concretises it.  As such
  we need to silence this warning.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• eb805c6c
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _options/optionbool.py: Refactor conditions
  
  The pylint tool prefers that conditions of the form "var == foo or var == bar"
  be refactored into "var in (foo, bar)".
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• aca9a612
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _platform/darwin.py: Remove useless __init__()
  
  An otherwise empty __init__ passing no arguments to super.__init__
  is worthless.  Remove it.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 77b1506f
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _platform/darwin.py: Remove unused imports
  
  The import of `resource` was unused.
  The import of `PlatformError` was unused.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 70000b0d
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _platform/linux.py: Remove unused imports
  
  The imports of `Message` and `MessageType` were unused.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• bab617d0
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _platform/linux.py: Simplify return statement
  
  The linter dislikes an if/return/else/return where it can be
  simplified to a return of the if condition.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 45a1d15f
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _scheduler/jobs/job.py: Correct lint issue around abstract method
  
  The linter cannot detect abstract methods the way we use them, so
  this silences the warning which ensues.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• e9154b4d
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _scheduler/queue/pullqueue.py: Remove unused return value
  
  The `done()` method does not return anything anywhere else.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 7b082bb7
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _scheduler/queue/trackqueue.py: Remove unused import
  
  The `SourceError` import was never used
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• f1cfc0b7
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  plugins/source/git.py: Rationalised unused return values
  
  The `assert_ref_in_track()` method's return value was never used so
  rationalise it to never return a value.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 7c72d25b
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  sandbox/_mount.py: Do not use dict literals in argument defaults
  
  The use of dictionary literals in argument defaults is disrecommended
  due to the way that they are static and thus potentially very confusing.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 4e7408ee
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  sandbox/_mounter.py: Remove useless inheritance on `object`
  
  In Python 3, inheriting from `object` explicitly is pointless.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• df495d16
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  sandbox/_sandboxbwrap.py: Disable lint about dict.get
  
  Sometimes `dict.get()` is preferable, in this instance it's less clear
  so we'll disable that lint
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 63c6e6b1
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  sandbox/_sandboxchroot.py: Silence warning about unused variable
  
  The variable was unused, remove it.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 3394741a
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  sandbox/_sandboxchroot.py: Silence warning about preexec_fn
  
  We are super-careful to not use threads in places where we might use
  Popen and as such this warning is save to quash.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• cd20e7a1
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  sandbox/_sandboxchroot.py: Silence warning about dict.get
  
  Sometimes `dict.get()` is preferable to an `if` statement, but this time
  it's clearer if we keep the structure.  As such, silence the warning.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 5087f5c5
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _artifactcache/cascache.py: Refactor slightly for indentation
  
  Correct some multi-line conditional indentation by slight refactor
  if necessary.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• d73d965b
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _artifactcache/cascache.py: Silence len(SEQUENCE) as condition warnings
  
  The lint tool dislikes the use of `if len(SEQ) == 0` type statements
  because nominally `len(SEQ)` may be more expensive than `bool(SEQ)`.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• fa740402
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _artifactcache/cascache.py: Prepare attributes in __init__
  
  Python linters prefer that attributes are defined in __init__ rather
  than left to later routines.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• ad8271f0
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _artifactcache/cascache.py: Remove unused imports
  
  The imports of `Message` and `MessageType` were unused.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 1445a901
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  _artifactcache/cascache.py: Cleanup lint warning about indexing
  
  In order to quieten the linter, and to make a more correct comparison,
  we switch to using the `Code.OK` enumeration from `google.rpc.Code`
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 19df07ea
  by Tristan Maat at 2018-10-25T14:43:52Z

  Bump CI image tags
  
  We need to do this so that the new linting errors pop up
  

• 64dbe9ae
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  MANIFEST: Include a lot more missing stuff in the manifest
  
  Versioneer needs to be in the MANIFEST.in and its .pyc needs to be ignored
  in the .gitignore.  Also much docs were not being included, nor conftest.py.
  Much of the test suite data files needed including, so changed that to a
  raw include of everything in the tests/ tree.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• f6bc25dd
  by Daniel Silverstone at 2018-10-25T14:43:52Z

  .gitlab-ci.yml: Change how coverage is recovered
  
  We need to alter how coverage data is recovered in order to work with
  newer coverage tooling.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 1c1d14e7
  by Chandan Singh at 2018-10-25T15:04:00Z

  Merge branch 'danielsilverstone-ct/fix-lint-issues' into 'master'
  
  Fix lint issues
  
  See merge request BuildStream/buildstream!892

• 23140b6b
  by Chandan Singh at 2018-10-25T15:25:22Z

  sources: Don't redundantly document directory configuration
  
  Instead of redundantly documenting "directory" configuration option in
  each source plugin, refer to the built-in functionality documentation of
  the Source class.
  
    * source.py: Add anchor for built-in functionality.
    * sources/plugins/*.py: Remove directory config option, and add reference to
      Source's built-in functionality.
  
  Fixes #713.
  

• db2a676e
  by Chandan Singh at 2018-10-25T15:26:39Z

  build elements: Add reference to BuildElement's built-in config
  
  For each element plugin derived from BuildElement, add reference to the
  built-in functionality documentation of BuildElement.
  
    * buildelement.py: Add anchor for built-in functionality.
    * plugins/elements/*.py: Add refence to BuildElement's built-in
      functionality for each build element.
  
  Related to #713.
  

• 356d84cd
  by Chandan Singh at 2018-10-25T16:59:13Z

  Merge branch 'chandan/source-doc-directory' into 'master'
  
  plugins: Don't redundantly document common configurations
  
  Closes #713
  
  See merge request BuildStream/buildstream!896

• 8c41b5b8
  by Daniel Silverstone at 2018-10-26T09:02:30Z

  _yaml.py: Do not insert into cache if retrieved from cache
  
  We were unconditionally inserting into the yaml cache anything we had
  retrieved from the yaml cache.  This had the side-effect of being expensive
  on re-runs of commands where yaml hadn't changed.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 494d7018
  by Jonathan Maw at 2018-10-26T09:26:36Z

  Merge branch 'danielsilverstone-ct/do-not-reinsert-yaml' into 'master'
  
  _yaml.py: Do not insert into cache if retrieved from cache
  
  See merge request BuildStream/buildstream!902

• 80762ecb
  by Benjamin Schubert at 2018-10-26T09:45:39Z

  Check if command is a str and replace by list before checking existence
  
  Currently we would make sure the sandbox had a command before
  converting it to a list if it was given as a string. That meant that a string
  command would never exist and the check be invalid.
  
  This also adds the same logic in the dummy sandbox for consistency.
  

• f131c407
  by Benjamin Schubert at 2018-10-26T10:09:51Z

  Merge branch 'bschubert/fix-command-sandbox' into 'master'
  
  Check is command is a str and replace by list before checking existence
  
  Closes #728
  
  See merge request BuildStream/buildstream!898

• 54d89aa1
  by Daniel Silverstone at 2018-10-26T10:20:36Z

  _loader/loadelement.py: Use enumerate() in dependency extraction
  
  In cases where elements end up with large numbers of dependencies,
  the n-squared performance in `_extract_depends_from_node` can produce
  significant delays during project load.  This patch corrects that.
  In addition we were double-retrieving provenance which was unnecessary.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• e4c18ba4
  by Phil Dawson at 2018-10-26T10:43:55Z

  Merge branch 'danielsilverstone-ct/use-enumerate' into 'master'
  
  _loader/loadelement.py: Use enumerate() in dependency extraction
  
  See merge request BuildStream/buildstream!901

• 20e5c58a
  by James Ennis at 2018-10-26T11:14:58Z

  using_configuring_artifact_server.rst: Fix lines that are too long
  

• 320dae02
  by James Ennis at 2018-10-26T11:37:42Z

  docs: Add more information to "managing the cache with systemmd" section
  

• 1eb993bf
  by James Ennis at 2018-10-26T12:02:49Z

  Merge branch 'jennis/another_docs_patch' into 'master'
  
  More information regarding the use of systemd to manage the cache
  
  See merge request BuildStream/buildstream!904

• f069d82f
  by Daniel Silverstone at 2018-10-26T12:19:58Z

  various: Move _sentinel from utils.py to _yaml.py
  
  The `_sentinel` in `utils.py` was used only for yaml node tracking.
  As such, simplify matters by removing it from `utils.py` and move it
  to `_yaml.py` which means that we no longer need to try and avoid a
  circular import issue by means of runtime importing.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 283887a5
  by James Ennis at 2018-10-26T13:12:12Z

  Merge branch 'danielsilverstone-ct/yaml-sentinel-rework' into 'master'
  
  Move _sentinel from utils.py to _yaml.py
  
  See merge request BuildStream/buildstream!903

• 2c6cb230
  by Valentin David at 2018-10-26T14:21:18Z

  Fix some ruamel warnings during tests
  

• a8250ca4
  by Valentin David at 2018-10-26T15:04:25Z

  Merge branch 'valentindavid/ruamel_warnings' into 'master'
  
  Fix some ruamel warnings during tests
  
  See merge request BuildStream/buildstream!889

• 1b308fe3
  by Daniel Silverstone at 2018-10-26T15:50:42Z

  _stream.py: Preserve stdout FD across checkout-to-stdout
  
  In order to support things which might need to use the stdout FD
  after checkout has completed writing a tarball to it, preserve it
  by means of `os.dup()` and `os.dup2()`
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 65d90c31
  by Daniel Silverstone at 2018-10-26T15:50:54Z

  tests/testutils/runcli.py: Support binary-mode capture of stdout
  
  In order to test things which write tarballs to stdout correctly,
  we need to capture the binary output cleanly.  This ensures we're not
  potentially mismatching encodings in and out.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• ed733f3e
  by Daniel Silverstone at 2018-10-26T15:51:40Z

  tests/frontend/buildcheckout.py: Enable test_build_checkout_tarball_stdout
  
  With the new support for saving the stdout FDs cleanly, and binary mode
  capture in runcli, enable the tarball-to-stdout test.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• e97d2a75
  by Benjamin Schubert at 2018-10-29T10:17:07Z

  Merge branch 'bschubert/fix-binary-output-capture' into 'master'
  
  Don't close stdout when running 'checkout' to stdout
  
  Closes #486
  
  See merge request BuildStream/buildstream!905

• 1625de0e
  by Tristan Van Berkom at 2018-10-29T11:49:33Z

  doc/source/conf.py: Added `images` directory for statically included files
  

• 38507b4e
  by Tristan Van Berkom at 2018-10-29T11:49:33Z

  doc/source/format_intro.rst: Adding link anchor for include directives
  

• 26164bcd
  by Tristan Van Berkom at 2018-10-29T13:57:51Z

  MANIFEST.in: Include SVG and ODG files in source distributions
  

• 3a23c8f6
  by Tristan Van Berkom at 2018-10-29T13:57:51Z

  doc: Adding new architecture document with initial "Overview of modules" section
  

• 41d370f6
  by Tristan Van Berkom at 2018-10-29T13:57:51Z

  doc: Adding new architecture document describing the data model
  

• bed7f7ac
  by Tristan Van Berkom at 2018-10-29T13:57:51Z

  doc: Adding new architecture document describing the dependency model
  

• 9bdf7f40
  by Tristan Van Berkom at 2018-10-29T13:57:51Z

  doc: Adding new architecture document about highlevel program flow
  

• 6b8bdf35
  by Tristan Van Berkom at 2018-10-29T13:57:51Z

  doc: Adding new architecture document about how the scheduler works.
  

• b1fadaa5
  by Tristan Van Berkom at 2018-10-29T13:57:51Z

  doc: Moved cachekey and sandboxing writings into the architecture section
  

• baf6b578
  by Tristan Van Berkom at 2018-10-29T14:28:41Z

  Merge branch 'tristan/architecture-docs' into 'master'
  
  Tristan/architecture docs
  
  See merge request BuildStream/buildstream!907

• 09ef8b25
  by Jonathan Maw at 2018-10-30T15:42:05Z

  element.py: Always clean up the rootdir
  
  We shouldn't need it to persist now that we cache failed build dirs.
  
  This change breaks the test
  `tests/integration/shell.py::test_sysroot_workspace_visible`.
  I can no longer see a use-case for this test.
  AIUI, it tested that the failed build sysroot stored in the builddir has
  the workspace's files in, despite the workspace being unmounted.
  I believe this behaviour is made redundant by cached buildtrees.
  
  This fixes part of #539
  

• 37afde0a
  by Jonathan Maw at 2018-10-30T15:42:05Z

  Element: Use cached buildtree in build shells and failure shells
  
  This includes changes in app.py:
  * Interactive failure shell no longer uses the failed build sysroot,
    defaulting to the cached build tree.
  
  Changes in element.py are:
  * Errors caused by building don't store the failed build sysroot,
    instead storing that a sandbox can be created to debug the error.
  * When staging sources, will stage the element's cached build tree if it
    exists.
  
  Changes in _exceptions.py:
  * BstError.sandbox is now a flag of whether a sandbox can be opened up
    to debug the error.
  
  Changes in widget.py:
  * Don't try to print any information about the sandbox.
  
  Changes in _message.py:
  * Fix documentation so Message.sandbox is not a directory any more.
  
  This is part of #539
  

• 4bc71773
  by Jonathan Maw at 2018-10-30T15:42:05Z

  NEWS: Add item for cached buildtree behaviour
  

• 9b89564f
  by Jonathan Maw at 2018-10-30T15:42:05Z

  tests: Add test that cached build trees are staged in build shells
  
  This is related to #539
  

• ec76cbe1
  by Jonathan Maw at 2018-10-30T15:42:05Z

  sandbox.py: Remove redundant Sandbox.__directory
  

• 825963b5
  by Jonathan Maw at 2018-10-30T15:42:05Z

  element: Make "--sysroot" take a bare directory
  
  i.e. instead of taking a directory that must contain "root" and
  "scratch", and treating "root" as the root, use the directory directly.
  
  In element.py:
  * __sandbox takes the `bare_sandbox` arg, to pass into the sandbox's constructor
  
  In sandbox.py:
  * If bare_sandbox, `_root` is the passed-in directory, and `__scratch`
    is None.
  * Trying to use `__scratch` when bare_sandbox is True is a bug.
  
  In _mount.py:
  * Don't get the value of `__scratch` if it's not needed.
  
  This is part of #539
  

• cf721838
  by Jonathan Maw at 2018-10-30T15:42:05Z

  NEWS: Add item for bst shell --sysroot changes
  

• 79d1bb7f
  by Jonathan Maw at 2018-10-30T15:42:05Z

  tests: Add tests for 'bst shell --sysroot'
  
  This is related to #539
  

• c7ac7e7d
  by Jonathan Maw at 2018-10-30T16:19:17Z

  Merge branch 'jonathan/debug-remote-failed-builds' into 'master'
  
  Jonathan/debug remote failed builds
  
  See merge request BuildStream/buildstream!869

• 516e990e
  by ctolentino8 at 2018-10-31T11:36:46Z

  _frontend/app.py: Set correct element-path in interactive bst-init
  

• b8a37a63
  by Tristan Van Berkom at 2018-11-01T10:16:25Z

  Merge branch 'chiaratolentino/bst-init-interactive-elementpath' into 'master'
  
  _frontend/app.py: Set correct element-path in interactive bst-init
  
  See merge request BuildStream/buildstream!912

• b27b592a
  by Benjamin Schubert at 2018-11-01T10:49:57Z

  Remove dependency on pytest-runner
  
  This includes a new command mimicking pytest-runner so that we
  can drop this dependency
  
  This was the only setup_requires dependency that we had and
  will make like easier for people behind proxies
  

• 89ace5d7
  by Benjamin Schubert at 2018-11-01T11:16:36Z

  Merge branch 'bschubert/remove-pytest-runner' into 'master'
  
  Remove dependency on pytest-runner
  
  See merge request BuildStream/buildstream!886

• 4cfabce8
  by Angelos Evripiotis at 2018-11-01T11:35:02Z

  BREAK: manual.yaml: don't set any default env vars
  
  Remove the 'MAKEFLAGS' and 'V' defaults.
  
  Now that there is a specialised 'make' element, these make-specific
  defaults don't make sense here. This element is meant to be a blank
  slate for folks to build on.
  
  Note that this is a breaking change, that will require some users to
  make changes to their .bst files if they are expecting these environment
  variable to be set.
  
  _versions.py: BST_FORMAT_VERSION bumped to 18.
  
  This fixes issue #718
  

• 48860aac
  by Tristan Van Berkom at 2018-11-01T12:01:04Z

  Merge branch 'tristan/empty-manual-element' into 'master'
  
  BREAK: manual.yaml: don't set any default env vars
  
  Closes #718
  
  See merge request BuildStream/buildstream!916

• d868b409
  by Daniel Silverstone at 2018-11-01T13:40:24Z

  _yaml.py: Implement `get()` for `ChainMap`
  
  Since the core Python `ChainMap.get()` implements with:
  
      self[key] if key in self else default
  
  The double-chain-lookup is expensive.  This simple change solves that
  for our ChainMap derived structure.  As such it improves matters
  for #466 somewhat.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 7f79b9ce
  by Tristan Van Berkom at 2018-11-01T14:25:57Z

  Merge branch 'danielsilverstone-ct/improve-chainmap-get' into 'master'
  
  _yaml.py: Implement `get()` for `ChainMap`
  
  See merge request BuildStream/buildstream!919

• de59ebdb
  by ctolentino8 at 2018-11-02T16:41:54Z

  plugins/sources/pip.py: Accomodate characters '-','.','_' for packages
  

• 8d7cf806
  by ctolentino8 at 2018-11-02T16:41:54Z

  tests/integration/project/files: Remove pypi-repo
  

• 9c2f9bf7
  by Chandan Singh at 2018-11-02T17:09:46Z

  Merge branch 'chiaratolentino/fix-pip-source-regex' into 'master'
  
  plugins/sources/pip.py: Accomodate characters '-','.','_' for packages
  
  See merge request BuildStream/buildstream!914

• 3788e701
  by Jürg Billeter at 2018-11-03T11:52:00Z

  element.py: Pass bare_directory also to SandboxRemote()
  
  Fixes: 825963b5 ("element: Make "--sysroot" take a bare directory")
  

• 82e971ef
  by Jürg Billeter at 2018-11-05T11:33:20Z

  Merge branch 'juerg/remote-execution' into 'master'
  
  element.py: Pass bare_directory also to SandboxRemote()
  
  See merge request BuildStream/buildstream!928

• 62942bfd
  by Valentin David at 2018-11-05T12:14:20Z

  Use less fetchers in overnight build to work-around limitations from servers.
  
  Issue happens with sourceware.org server.
  
  Fixes #742
  

• 442da2f9
  by Javier Jardón at 2018-11-05T12:41:54Z

  Merge branch 'valentindavid/limit_fetchers_in_overnight_build' into 'master'
  
  Use less fetchers in overnight build to work-around limitations from servers.
  
  Closes #742
  
  See merge request BuildStream/buildstream!929

• 0993de53
  by Richard Maw at 2018-11-05T16:41:05Z

  dev-requirements: Update pytest dependency to 3.8
  
  "_frontend/app.py: Set correct element-path in interactive bst-init"
  added a dependency on the tmp_path test fixture, which was added by 3.8.
  

• be8f0a54
  by richardmaw-codethink at 2018-11-05T17:08:43Z

  Merge branch 'richardmaw/pytest-dep-update' into 'master'
  
  dev-requirements: Update pytest dependency to 3.8
  
  See merge request BuildStream/buildstream!930

• bfb639bf
  by Jürg Billeter at 2018-11-05T17:18:12Z

  element.py: Remove artifacts parameter from constructor
  
  Get the artifact cache from the context instead.
  

• ca855f91
  by Jürg Billeter at 2018-11-05T17:18:12Z

  _elementfactory.py: Remove unused artifacts parameter from create()
  

• 15fed21c
  by Jürg Billeter at 2018-11-05T17:18:12Z

  _project.py: Remove unused artifacts parameter from create_element()
  

• 0085d2aa
  by Jürg Billeter at 2018-11-05T17:18:12Z

  element.py: Remove unused artifacts parameter from _new_from_meta()
  

• f69b1117
  by Jürg Billeter at 2018-11-05T17:18:12Z

  _project.py: Remove unused artifacts parameter from load_elements()
  

• e398f877
  by Jürg Billeter at 2018-11-05T17:18:12Z

  element.py: Do not include type name of artifact cache in cache key
  
  The artifact cache backend does not affect build outputs and we anyway
  no longer have pluggable artifact cache backends. This hardcodes
  CASCache instead of removing the entry completely to avoid cache key
  changes.
  

• 626d20ae
  by Jürg Billeter at 2018-11-05T17:18:12Z

  Split up artifact cache and CAS cache
  
  This changes CASCache from a subclass to a delegate object of
  ArtifactCache. As the lower layer, CASCache no longer deals with
  elements or projects.
  
  Fixes #659.
  

• ec04446b
  by Jürg Billeter at 2018-11-05T17:51:39Z

  Merge branch 'juerg/cas' into 'master'
  
  Split up artifact cache and CAS cache
  
  Closes #659
  
  See merge request BuildStream/buildstream!922

• 9a045080
  by Josh Smith at 2018-11-06T13:23:19Z

  utils: Fix _pretty_size() for sizes > 1024T
  

• d3a07e6b
  by Josh Smith at 2018-11-06T13:23:19Z

  Add regression test for _pretty_size
  

• 0c09fb9c
  by richardmaw-codethink at 2018-11-06T14:12:20Z

  Merge branch 'Qinusty/unit-test-utils' into 'master'
  
  Fix issue with _pretty_size with large numbers of bytes
  
  See merge request BuildStream/buildstream!799

• b4eec489
  by Jim MacArthur at 2018-11-06T16:44:14Z

  _sandboxremote.py: Use the standard SandboxError.
  
  Replaces the custom one which was erroneously added during development.
  Fixes #746.
  

• 1f7acf74
  by Jim MacArthur at 2018-11-06T17:16:31Z

  Merge branch 'jmac/remote-sandbox-errors' into 'master'
  
  _sandboxremote.py: Use the standard SandboxError
  
  Closes #746
  
  See merge request BuildStream/buildstream!931

• 8e64ccef
  by James Ennis at 2018-11-07T11:06:11Z

  using_config.rst: Update our instructions for how we declare caches in user config
  

• 09ab676d
  by James Ennis at 2018-11-07T11:06:11Z

  format_project.rst: Update instructions regarding how we declare remote caches in project.conf
  

• f514124f
  by James Ennis at 2018-11-07T11:06:11Z

  using_configuring_artifact_server.rst: Remove examples and redirect to user config/project config
  

• 029ba17d
  by richardmaw-codethink at 2018-11-07T11:32:20Z

  Merge branch 'jennis/docs_patch' into 'master'
  
  Update our instructions/examples regarding how we declare remote caches
  
  See merge request BuildStream/buildstream!932

• 261e2cd3
  by Jim MacArthur at 2018-11-07T12:02:43Z

  element.py: Use _force_rmtree instead of custom code.
  
  There are arguments for and against using onerror - see issue #735
  for details.
  

• 8931e42c
  by Jim MacArthur at 2018-11-07T12:26:06Z

  Merge branch 'jmac/make_writable_combination' into 'master'
  
  Use _force_rmtree instead of custom code.
  
  Closes #735
  
  See merge request BuildStream/buildstream!927

• 6ccfab0b
  by Valentin David at 2018-11-07T16:28:47Z

  Fix cache corruption by scripts when layout and integration commands are used
  
  Root directory was marked as a non-artifact mount, so not using
  SafeHardLink.  However integration commands executed with write access
  to the root directory.
  
  Fixes #749
  

• f24e20e9
  by richardmaw-codethink at 2018-11-07T16:59:50Z

  Merge branch 'valentindavid/script-artifact-corruption' into 'master'
  
  Fix cache corruption by scripts when layout and integration commands are used
  
  Closes #749
  
  See merge request BuildStream/buildstream!934

• e28f668a
  by Valentin David at 2018-11-08T09:34:11Z

  Fix infinite recursion in default strip debug command
  
  Fixes #645.
  

• 83d15350
  by Valentin David at 2018-11-08T10:01:39Z

  Merge branch 'valentindavid/fix-recursive-strip-debug' into 'master'
  
  Fix infinite recursion in default strip debug command
  
  Closes #645
  
  See merge request BuildStream/buildstream!918

• 9f0e12f1
  by Benjamin Schubert at 2018-11-08T10:21:12Z

  Move bwrap checks in platform/linux.py
  
  Remove the bwraps checks from _site.py and put them in platform.linux
  which is the only place where they are run.
  
  This allows the removal of a double level of caching, making
  reasoning about tests easier
  

• cf2e0059
  by Benjamin Schubert at 2018-11-08T10:21:12Z

  conftest.py: Ensure platform is not maintained between tests
  
  This removes the `_instance` on the platform object that we use
  for caching and not recreating the object everytime at the start
  of every test.
  
  This is to ensure our tests share the least amount of state.
  
  The performance penalty is from 5 to 10% accross the whole test suite.
  The readings were done 5 times for each before and after the change
  and on the same computer.
  

• c51ba01b
  by Benjamin Schubert at 2018-11-08T10:21:12Z

  Test that helpful messages are raised when missing dependencies
  
  This adds a `reason` to the SandboxEror thrown in sandboxdummy
  to be able to understand where the error comes from
  

• f116b9b7
  by Benjamin Schubert at 2018-11-08T11:07:12Z

  Merge branch 'bschubert/tests-no-chroot-linux' into 'master'
  
  test behavior when there is no sandbox on Linux
  
  Closes #696 and #736
  
  See merge request BuildStream/buildstream!921

• cc2e6ae5
  by Valentin David at 2018-11-08T12:41:36Z

  Fix bug with root mounted as non-artifact in script plugin.
  
  The issue was introduced by 6ccfab0b1b25990e406446d5cbe5aee83a5e158a.
  

• e578a89f
  by Valentin David at 2018-11-08T13:11:10Z

  Merge branch 'valentindavid/fix-script-install-root' into 'master'
  
  Fix bug with root mounted as non-artifact in script plugin.
  
  Closes #756 and #749
  
  See merge request BuildStream/buildstream!941

• c0a8bb66
  by Angelos Evripiotis at 2018-11-08T15:49:16Z

  contributing: WIP again to be kind to reviewers
  
  As someone coming from GitHub to GitLab, I was pleasantly surprised by
  the 'filter by WIP status' option.
  
  To make sure we get the most out of it, add a guideline to keep the
  filter clean, to reduce reviewer burden.
  

• f7231e90
  by Angelos Evripiotis at 2018-11-08T15:49:16Z

  contributing: non-WIP should always be landable
  
  I was recently surprised that we don't prefer addressing review comments
  in 'fixup!' commits.
  
  Coming from GitHub, I've found that fixup commits make it easier for
  reviewers to see what has changed since their last review. The idea is
  to use '--autosquash' before landing to clean up the history again.
  
  It's a pleasant surprise that it's easy to keep track of what changed in
  merge-requests between pushes, so we can always keep the history clean.
  
  Document this, so folks like me can see the light sooner.
  

• f7643440
  by Angelos Evripiotis at 2018-11-08T15:49:16Z

  contributing: emphasise the 'why' in commits
  
  Explain why it's useful to mention the decisions made for a change and
  provide links for background, e.g. issue numbers.
  
  My general experience of git histories is that folks don't do enough
  explaining, so it's great we have a contributing section for it.
  
  I've found it's easier for folks to stick to rules when they know the
  practical reasons for them, so provide some.
  

• dd5e7b04
  by Angelos Evripiotis at 2018-11-08T16:17:04Z

  Merge branch 'aevri/contributing_gitlab' into 'master'
  
  Add more to GitLab-relevant parts of contributing
  
  See merge request BuildStream/buildstream!935

• f03ed2d6
  by Chandan Singh at 2018-11-12T10:33:59Z

  Ensure `--deps=none` option works for `bst checkout`
  
  Currently, `bst checkout --deps none` command always produces empty
  output. Fix this issue and add regression test for the same.
  
  - element_enums.py: Add Scope.NONE.
  
  - element.py: Ensure Scope.NONE works correctly in addition to
    Scope.RUN/Scope.ALL in Element.dependencies() and Element.search().
  
  - tests/frontend/buildcheckout.py: Fix tests for `--deps none`.
  
  Fixes #670.
  

• 78e7e1c3
  by Chandan Singh at 2018-11-12T10:33:59Z

  Add `--deps build` option to `bst checkout`
  
  As discussed in
  https://mail.gnome.org/archives/buildstream-list/2018-September/msg00064.html,
  add `--deps build` option to `bst checkout`. This will allow users to
  checkout the all build dependencies of a given element using a single
  command.
  
  - _frontend/cli.py: Add `--deps build` option for `bst checkout`.
  
  - element.py: Support `deps='build'` in Element._prepare_sandbox().
  
  - tests/frontend/buildcheckout.py: Ensure `--deps build` works as
    expected.
  

• 9f0cb743
  by Benjamin Schubert at 2018-11-12T10:34:00Z

  Refactor and simplify _prepare_sandbox for elements
  
  Before we would have a intricate logics with multiple arguments
  that might get ignored.
  
  This simplifies the design and introduces a bool `shell` instead
  of having two different variables concerned about scope
  

30 changed files:

• .gitignore
• .gitlab-ci.yml
• CONTRIBUTING.rst
• MANIFEST.in
• NEWS
• README.rst
• buildstream/_artifactcache/artifactcache.py
• buildstream/_artifactcache/cascache.py
• buildstream/_artifactcache/casserver.py
• buildstream/_context.py
• buildstream/_elementfactory.py
• buildstream/_exceptions.py
• buildstream/_frontend/app.py
• buildstream/_frontend/cli.py
• buildstream/_frontend/status.py
• buildstream/_frontend/widget.py
• buildstream/_fuse/hardlinks.py
• buildstream/_fuse/mount.py
• buildstream/_includes.py
• buildstream/_loader/loadelement.py
• buildstream/_loader/loader.py
• buildstream/_message.py
• buildstream/_options/optionbool.py
• buildstream/_options/optionpool.py
• buildstream/_pipeline.py
• buildstream/_platform/darwin.py
• buildstream/_platform/linux.py
• buildstream/_project.py
• buildstream/_scheduler/jobs/job.py
• buildstream/_scheduler/queues/pullqueue.py

The diff was not included because it is too large.

—
View it on GitLab.
You're receiving this email because of your account on gitlab.com. If you'd like to receive fewer emails, you can adjust your notification settings.