[Notes] [Git][BuildStream/buildstream][chandan/junction-dependency-format] 81 commits: _yaml.py: Only retrieve provenance in node_get() when needed

[Chandan Singh <gitlab mg gitlab com>]

Title: GitLab

Chandan Singh pushed to branch chandan/junction-dependency-format at BuildStream / buildstream

Commits:

• a39c4767
  by Daniel Silverstone at 2019-02-12T15:12:30Z

  _yaml.py: Only retrieve provenance in node_get() when needed
  
  We were indiscriminately retrieving the node's provenance data in the
  `node_get()` function which was accounting for approximately a third of
  the total runtime of `node_get()` which dominates pre-scheduler time in
  `bst build`.  This change ameliorates that situation by only retrieving
  the provenance data when it's actually needed.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• cda03313
  by Daniel Silverstone at 2019-02-12T15:12:30Z

  _yaml.py: Reduce use of `isinstance()` in `node_sanitize()`
  
  We know that nodes are typically one of:
  str, list, dict, bool, tuple, NoneType or our ChainMap
  Of these, dict and ChainMap are Mapping, only list is list
  and the rest are returned unchanged.  We can reduce/defer our use
  of isinstance here, dramatically, improving performance.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• b6f08e1b
  by Daniel Silverstone at 2019-02-12T15:12:30Z

  _yaml.py: Remove use of isinstance() in `{node,list}_{chain_,}copy`
  
  A non-trivial proportion of time pre-scheduler in `bst build` is spent
  copying (or chain-copying) nodes.  Approximately a quarter of the time
  spent in that effort is in `isinstance()`.  This removes that CPU load.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 69ee11c6
  by Daniel Silverstone at 2019-02-12T15:12:30Z

  _yaml.py: Reduce cost of node_final_assertions
  
  By re-using the isinstance replacements from earlier commits and
  using a tuple of the string constants for checking for composition
  markers, we reduce the cost of node_final_assertions by two thirds
  in basic testing.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 0928e570
  by Daniel Silverstone at 2019-02-12T15:12:30Z

  _context.py: Cache result of get_strict()
  
  While get_strict() doesn't look expensive per-se, it is called so many
  times that it is valuable to cache the result once computed.  Since I
  don't think it can change once it is computable, cache it immediately
  that becomes possible and we save 20s in my test case.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 022a59f0
  by Benjamin Schubert at 2019-02-12T16:14:05Z

  Merge branch 'danielsilverstone-ct/further-optimisations' into 'master'
  
  Further optimisations
  
  See merge request BuildStream/buildstream!1131

• 352f4ad9
  by Jonathan Maw at 2019-02-12T16:44:35Z

  loader.py: Stage junctions into .bst instead of a tmpdir
  

• 62396af9
  by Jonathan Maw at 2019-02-12T16:44:35Z

  loader.py: Load junction from workspace if one's open
  

• f1e9cb66
  by Jonathan Maw at 2019-02-12T16:44:35Z

  Remove cleanup functionality from loader
  
  Since junction check-outs are now persistent, there is no need to clean
  up temporary directories once finished.
  Now, junction checkouts are stored within the top-level project dir and
  will be cleaned-up when that directory is removed.
  
  This involves changes in:
  * loader.py: Remove cleanup logic and passing-around of tmpdir.
  * _project.py: Remove passing around tmpdir and calling loader's
                 cleanup.
  

• e51116d5
  by Jonathan Maw at 2019-02-12T17:59:25Z

  Merge branch 'jonathan/junction-no-tmpdir' into 'master'
  
  Stage junctions into .bst instead of a tmpdir
  
  Closes #895
  
  See merge request BuildStream/buildstream!1134

• 478e5c47
  by Valentin David at 2019-02-12T18:06:50Z

  buildstream/_cas/cascache.py: Set 0644 rights to pulled files
  
  This was broken by 5ef19a0b31df84caed1e41719ef7ea5c6bd8a8bc.
  

• 6de65306
  by Valentin David at 2019-02-12T19:13:43Z

  Merge branch 'valentindavid/pull-chmod-bug' into 'master'
  
  buildstream/_cas/cascache.py: Set 0644 rights to pulled files
  
  See merge request BuildStream/buildstream!1144

• 9b140fa0
  by James Ennis at 2019-02-13T09:35:45Z

  element.py: Lift ArtifactCache.get_artifact_fullname() to here
  
  This commit removes the method ArtifactCache.get_artifact_fullname()
  and replaces it with Element.get_artifact_name()
  
  Given a key, we are now able to construct the full name of any of an
  element's artifacts.
  

• 039d7c1d
  by James Ennis at 2019-02-13T09:35:45Z

  element.py: Make calculate_cache_key() API private
  

• 7cb6be74
  by James Ennis at 2019-02-13T09:35:45Z

  element.py: Return early in __init_defaults if no plugin config
  

• ddf642e1
  by James Ennis at 2019-02-13T09:35:45Z

  metaelement.py: Provide constructor with default keyword arguments
  

• 55c15f8f
  by James Ennis at 2019-02-13T09:35:45Z

  _artifactelement.py: New ArtifactElement object (derived from Element)
  
  This object should be used when we want to handle artifact refs directly
  from the command line.
  An ArtifactElementError has also been added to _exceptions.py
  

• 6951cfc8
  by James Ennis at 2019-02-13T09:35:45Z

  _project.py: Add create_artifact_element() method
  

• bf1933b6
  by James Ennis at 2019-02-13T09:35:45Z

  _stream.py: Add the _classify_artifacts() helper
  

• ea6ff6da
  by James Ennis at 2019-02-13T09:35:45Z

  _stream.py: Modify behaviour of _classify_artifacts()
  
  * There is no need for this method to use a cas object.
  * Search for artifact globs in the project's element path
  * An artifact key is always 64 chars long
  

• fcc79917
  by James Ennis at 2019-02-13T09:35:45Z

  _stream.py: Allow loading to handle artifact refs
  

• 77345317
  by James Ennis at 2019-02-13T09:35:45Z

  _artifactcache.py: Add get_artifacts_log() method
  
  A CasBasedDirectory object of an artifacts logs can be obtained
  with ArtifactCache.get_artifacts_log(). This ultimately calls
  CASCache.get_top_level_dir() to obtain a CasBasedDirectory
  of an artifact's subdirectory (or subdirectories).
  

• 775ac472
  by James Ennis at 2019-02-13T09:35:45Z

  cli.py: Move artifact ref handling logic to stream
  
  The loading of elements and the handling of artifacts does not belong
  in this module. Such logic should be invoked using the Stream API
  

• f95d6ee8
  by James Ennis at 2019-02-13T09:35:45Z

  cli.py: Remove _classify_artifacts() method
  
  This method has been moved to Stream, where is it used there exclusively.
  

• 491937d7
  by James Ennis at 2019-02-13T09:35:45Z

  cascache.py: Allow CASCache.list_refs() to handle globs
  
  This commit ensures that CASCache.list_refs(), and
  ArtifactCache.list_artifacts(), can both handle glob expressions.
  

• 6ae41474
  by James Ennis at 2019-02-13T09:35:45Z

  _stream.py: _classify_artifacts() should be able to handle globs
  
  _classify_artifacts() no longer filters glob expressions by first
  obtaining a list of ALL refs locally cached. We now only obtain refs
  specified by the glob _expression_.
  
  Furthermore, the Project.element_path is used to start searching for
  globbed elements as opposed to the Project.directory.
  

• 8c6baf23
  by James Ennis at 2019-02-13T10:48:21Z

  Merge branch 'jennis/refactor_artifact_log' into 'master'
  
  Refactor artifact log command
  
  See merge request BuildStream/buildstream!1101

• 14db3287
  by Jürg Billeter at 2019-02-13T10:48:45Z

  projectconfig.yaml: Consistently include directories in split rules
  
  Most split rules already included the relevant directories themselves in
  addition to the directory contents.  Add the missing bin, sbin, and
  libexec directories.
  
  This is required to fix tests with the following commit that changes
  list_relative_paths() to return all directories.
  

• c7625cdf
  by Jürg Billeter at 2019-02-13T10:48:45Z

  tests/integration/project: Add tests directory to split rule
  
  This is required to fix tests with the following commit that changes
  list_relative_paths() to return all directories.
  

• 925429ea
  by Jürg Billeter at 2019-02-13T10:48:45Z

  utils.py: Return all directories in list_relative_paths()
  
  Returning only empty directories leads to inconsistencies when computing
  a manifest by combining results from multiple list_relative_paths()
  calls as done by the compose plugin.
  
  I.e., the same directory may be empty in one dependency and non-empty in
  another dependency.  The merged file list will still contain that
  directory even though it's no longer empty.
  
  This inconsistency causes problems when calculating differences between
  manifests. Returning all directories fixes these inconsistencies.
  
  This is a change in API behavior.
  

• 353745a4
  by Jürg Billeter at 2019-02-13T10:48:45Z

  _casbaseddirectory.py: Return all directories in list_relative_paths()
  
  This matches the change in utils.list_relative_paths().
  

• ae0ee361
  by Jürg Billeter at 2019-02-13T12:42:42Z

  Merge branch 'juerg/list-all-directories' into 'master'
  
  Return all directories in list_relative_paths()
  
  See merge request BuildStream/buildstream!1139

• 118644b2
  by Tom Pollard at 2019-02-13T13:37:32Z

  Add cli main & user conf option for 'cache-buildtrees' context
  
  _context.py: Add cache_buildtrees global user context, the default
  of which is set to by default to 'always' via the addition of
  cache-buildtrees to userconfig.yaml cache group. 'failure' & 'never'
  can be given as valid options.
  
  app.py & cli.py: Add --cache-buildtrees as a bst main option, which
  when passed with a valid option can override the default or user
  defined context for cache_buildtrees.
  
  tests/completions/completions.py: Update for the added flag.
  

• b546bac1
  by Tom Pollard at 2019-02-13T13:37:32Z

  _frontend/cli.py: Ensure failed buildtree warning is correct
  
  not _cached_sucess() could resolve to true if the element wasn't
  cached at all. switch to _cached_failure() to ensure condition
  reflects expected artifact state
  

• d2029444
  by Tom Pollard at 2019-02-13T13:37:32Z

  Provide configuration for the optional creation of buildtrees
  
  Artifacts can be cached explicitly with an empty `build tree` when
  built via the cli main options or user config for all or only
  successful build artifacts. Default behaviour is to still create
  and cache all expected buildtrees.
  
  element.py: _cache_artifact() Check if context for cache_buildtrees
  has been set to always or failure with a corresponding build
  result, if not skip attempting to export the build-root. Element
  types without a build-root are cached with an empty buildtree
  regardless. Update _stage_sources_at() to warn the user that the
  buildtree import is empty.
  
  tests/integration: Add test to artifact.py for the optional caching
  of buildtree content from bst build. Rename build-tree.py to
  shellbuildtrees.py to reflect included test cases, add test for
  empty buildtree warning and failure option.
  
  NEWS: Add entry for new option.
  

• 54ec032a
  by Jürg Billeter at 2019-02-13T14:50:15Z

  Merge branch 'tpollard/896' into 'master'
  
  Optional creation of buildtrees
  
  Closes #896
  
  See merge request BuildStream/buildstream!1135

• ec4bbf35
  by Benjamin Schubert at 2019-02-13T14:59:47Z

  Cleanup internal Loader cache after loading elements
  

• b41a82d3
  by Benjamin Schubert at 2019-02-13T14:59:47Z

  Don't register exceptions when not running the testsuite
  
  This fix a problem with the garbage collector not being able to
  clean the MetaElements that are loaded.
  
  On small projects this is not a problem, but in bigger projects, this
  can save a few hundred of MBs at runtime
  
  The reason behind this is, whenever we have a "stack" element, which
  has no stack.yaml configuration, since it doesn't need it, we would
  get an exception thrown when initiating the first one, as loading the
  yaml file would fail.
  
  This would capture the frame in which this command was executed, which
  references meta_elements. Therefore, as long as another exception is not
  thrown, the garbage collector would not be able to clean all the
  MetaElements.
  

• 9db7f489
  by Benjamin Schubert at 2019-02-13T16:13:16Z

  Merge branch 'bschubert/cleanup-local-state' into 'master'
  
  Cleanup MetaElement local state
  
  See merge request BuildStream/buildstream!1147

• c2c10d68
  by Daniel Silverstone at 2019-02-14T13:02:07Z

  Cache Keys: Update to use JSON rather than pickle
  
  This affects the cache key version (updated to 7) and introduces
  a dependency on `ujson` which is BSD licenced as of the version
  locked in `requirements.txt`
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 868c057e
  by Tristan Van Berkom at 2019-02-14T13:58:39Z

  Merge branch 'danielsilverstone-ct/json-cache-key' into 'master'
  
  Update cache keys to use JSON
  
  See merge request BuildStream/buildstream!1151

• ace15fa6
  by Benjamin Schubert at 2019-02-14T14:04:45Z

  Don't keep MetaSource around in Source
  
  We can easily rebiuld MetaSource from any source so there is no
  reason to keep them around.
  
  This will slightly improve memory usage.
  

• 174f5394
  by Tristan Van Berkom at 2019-02-14T15:04:27Z

  Merge branch 'bschubert/dont-keep-metasource' into 'master'
  
  Don't keep MetaSource around in Source
  
  See merge request BuildStream/buildstream!1150

• e7e4086e
  by Jürg Billeter at 2019-02-14T15:05:38Z

  utils.py: Change _ensure_real_directory() to not resolve symlinks
  
  Resolving symlinks during staging causes various issues:
  
  * Split rules may not work properly as the resolved paths will differ
    depending on whether another artifact with a directory symlink has
    been staged in the same root directory or not, e.g., as part of
    compose.
  
  * The order of symlinks in file lists is difficult to get right to
    guarantee consistent and predictable behavior as paths in a file list
    might rely on symlinks in the same file list. See #647 and #817.
  
  * Staging order differences can lead to surprising results. See #390.
  
  * Difficult to properly support absolute symlinks. Absolute symlinks are
    currently converted to relative symlinks, however, this doesn't always
    work. See #606 and #830.
  
  This will require changes in projects that rely on the current behavior.
  However, the changes are expected to be small and are often a sign of
  buggy element files. E.g., elements that don't fully obey `bindir` or
  `sbindir` variables.
  

• c0570871
  by Jürg Billeter at 2019-02-14T15:05:38Z

  _casbaseddirectory.py: Do not resolve symlinks
  
  This matches the change in utils._process_list().
  
  This also removes the _Resolver class as it is now unused. We may want
  to support controlled symlink resolution in the future, in which case
  the _Resolver class can be resurrected from this commit.
  

• 55bccf63
  by Jürg Billeter at 2019-02-14T15:05:38Z

  utils.py: Do not mangle absolute symlinks
  
  Copy symlinks as they are, absolute or relative. We no longer resolve
  symlinks when copying files, which makes this safe.
  

• 94c6d59b
  by Jürg Billeter at 2019-02-14T15:05:38Z

  Bump artifact version for changes in symlink handling
  

• 949962f8
  by Jürg Billeter at 2019-02-14T15:05:38Z

  NEWS: Add entry for change in symlink handling
  

• 56c07baa
  by Jürg Billeter at 2019-02-14T16:08:42Z

  Merge branch 'juerg/symlinks2' into 'master'
  
  Do not resolve or mangle symlinks during staging
  
  See merge request BuildStream/buildstream!1140

• 69520011
  by Chandan Singh at 2019-02-15T06:58:27Z

  contrib/bst-graph: Add script to print graph in DOT format
  
  This script leverages the recently added format strings
  (`%{build-deps}`, `%{runtime-deps}`) to `bst show` to print a graph in
  DOT format. This requires users to have the `graphviz` python package
  installed.
  
  Additionally, users can also render the graph using the `--format`
  option if they have the `graphviz` command line tool installed.
  

• 51dae747
  by Chandan Singh at 2019-02-15T08:50:21Z

  Merge branch 'chandan/dot-graph' into 'master'
  
  contrib/bst-graph: Add script to print graph in DOT format
  
  Closes #705
  
  See merge request BuildStream/buildstream!1148

• 0f78a47d
  by Daniel Silverstone at 2019-02-15T10:03:46Z

  Variables: Rework how expansion strings work
  
  Rather than constantly using regular expressions and retrieval from YAML
  nodes, pre-parse expansion strings into a list representation cached
  for reuse, and then expand them as simple string concatenation.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 14176e51
  by Benjamin Schubert at 2019-02-15T11:04:10Z

  Merge branch 'danielsilverstone-ct/variables-rework' into 'master'
  
  Variables: Rework how expansion strings work
  
  See merge request BuildStream/buildstream!1152

• 39febfda
  by Angelos Evripiotis at 2019-02-15T11:10:51Z

  storage.Directory.export_to_tar: mtime=_magic...
  
  Change the default value of mtime when doing export_to_tar() from `0` to
  `_utils._magic_timestamp`. This avoids problems in other software, which
  assume that an mtime of `0` means the file does not exist.
  
  There is more than one example where files with an mtime of zero are
  treated as non-existant by other software. e.g. [ninja][1] and [Template
  Toolkit][2].
  
  The OSTree project also [express an intention][3] to move from an mtime
  of 0 to an mtime of 1:
  
  > For this reason, OSTree acts as though all timestamps are set to
  > time_t 0, so that comparisons will be considered up-to-date. Note that
  > for a few releases, OSTree used 1 to fix warnings such as GNU Tar
  > emitting "implausibly old time stamp" with 0; however, until we have a
  > mechanism to transition cleanly to 1, for compatibilty OSTree is
  > reverted to use zero again.
  
  >From the comments on export_to_tar(), the motivation for having an mtime
  of 0 was to have reproducible results, rather than it specifically being
  the value 0.
  
  Additionally, the reproducible builds project has a [page on
  archives][4]; it mentions the benefits of setting all the files to have
  the same mtime, or clamping the mtime. It makes no mention of a
  motivation for the mtime to be specifically 0.
  
  Fixes #914
  
  [1]: https://github.com/ninja-build/ninja/issues/1120
  [2]: https://github.com/abw/Template2/blob/8d7d37200af436f1ad43628278d3caad257c8e27/lib/Template/Provider.pm#L635
  [3]: https://ostree.readthedocs.io/en/latest/manual/repo/
  [4]: https://reproducible-builds.org/docs/archives/
  

• 8b34e356
  by Angelos Evripiotis at 2019-02-15T12:25:21Z

  Merge branch 'aevri/mtime1' into 'master'
  
  storage.Directory.export_to_tar: default mtime=utils._magic_timestamp
  
  Closes #914
  
  See merge request BuildStream/buildstream!1149

• 3bbb90e7
  by Benjamin Schubert at 2019-02-15T12:34:36Z

  Use sets when checking for existence of an element
  

• 0921ccf4
  by Benjamin Schubert at 2019-02-15T13:39:52Z

  Merge branch 'bschubert/set-as-set' into 'master'
  
  Use sets when checking for existence of an element
  
  See merge request BuildStream/buildstream!1154

• c24f2971
  by Angelos Evripiotis at 2019-02-15T13:56:48Z

  userconfig: rm really-workspace-close-project-inaccessible
  
  Remove the need for the 'really-workspace-close-project-inaccessible'
  config option, as well as the option itself.
  
  As agreed on the mailing list [1], all the 'are you sure?' prompts on
  workspace reset and close were removed. While that discussion was going
  on, this new prompt and option was added. At the 2019 BuildStream
  Gathering, it was verbally agreed between myself and Tristan VB that we
  would also remove this instance.
  
  It was also agreed that we should have a notice to let the user know
  what they'd done, this was already in place if interactive. Moved it to
  be unconditional so that there's no difference in non-interactive
  behaviour. Made it output to stderr, as it's diagnostic meant for the
  user. Made it the last thing echo'd so it's next to the prompt - it's
  very relevant to what they type next. Added a test to make sure the text
  makes it to stderr in the appropriate case, and not in an inappropriate
  one.
  
  This is the last instance of any prompt configuration, so BuildStream
  can also forget all of that machinery.
  
  [1] https://mail.gnome.org/archives/buildstream-list/2018-December/msg00111.html
  

• 91eb29a6
  by Angelos Evripiotis at 2019-02-15T14:55:06Z

  Merge branch 'are_you_sure2' into 'master'
  
  userconfig: rm really-workspace-close-project-inaccessible
  
  Closes #726 and #744
  
  See merge request BuildStream/buildstream!1130

• b8561fb1
  by Chandan Singh at 2019-02-15T14:58:22Z

  contrib/bst-here: Change default image to buildstream/buildstream
  
  See https://gitlab.com/BuildStream/buildstream-docker-images/issues/26
  for detailed discussion around this.
  
  `buildstream/buildstream-fedora` is now considered deprecated. Switch to
  `buildstream/buildstream` image. This image also offers more tags that
  will provide users more flexibility.
  

• 2699c77c
  by Chandan Singh at 2019-02-15T14:58:31Z

  contrib/bst-here: Allow users to specify image variant
  
  Now that the `buildstream/buildstream` image has 9 variants, let's make
  it easier to choose the desired tag, using a command-line option.
  
  This is otherwise possible by specifying the full image name `image:tag`
  using the `-i` option. But, this will make it easier to specify just the
  tag using `-j`.
  
  The following two invocations of `bst-here` are now equivalent:
  
      bst-here -i buildstream/buildstream:dev
      bst-here -j dev
  

• 677fc6c5
  by Chandan Singh at 2019-02-15T16:09:21Z

  Merge branch 'chandan/bst-here-tags' into 'master'
  
  contrib/bst-here: Allow users to specify image variant
  
  See merge request BuildStream/buildstream!1153

• 3b889eb8
  by Javier Jardón at 2019-02-15T20:22:08Z

  README.rst: Add table with distros with packaged buildstream
  
  [ci skip]
  

• 232906b1
  by Javier Jardón at 2019-02-15T23:08:50Z

  Merge branch 'jjardon/distros' into 'master'
  
  README.rst: Add table with distros with packaged buildstream
  
  See merge request BuildStream/buildstream!1143

• ee2296ab
  by Tristan Van Berkom at 2019-02-18T08:41:57Z

  CONTRIBUTING.rst: Documenting the release process
  
  So that other people can also make releases.
  

• c79696fe
  by Tristan Van Berkom at 2019-02-18T09:43:22Z

  Merge branch 'tristan/document-release-process' into 'master'
  
  CONTRIBUTING.rst: Documenting the release process
  
  See merge request BuildStream/buildstream!1155

• fb65af6c
  by Jürg Billeter at 2019-02-18T10:29:25Z

  _casbaseddirectory.py: Fix order in _recalculate_recursing_up()
  
  The directory needs to be serialized after the Digest for the
  subdirectory `caller` has been updated.
  

• f5660fa0
  by Jim MacArthur at 2019-02-18T10:29:25Z

  _casbaseddirectory.py: Correct typo for _get_underlying_directory
  

• cdcf0dc5
  by Jürg Billeter at 2019-02-18T10:29:25Z

  _casbaseddirectory.py: Remove unused find_root() method
  

• f9dd6ea2
  by Jim MacArthur at 2019-02-18T10:29:25Z

  storage: Add Directory.get_size()
  

• 050249bb
  by Jürg Billeter at 2019-02-18T10:29:25Z

  _casbaseddirectory.py: Add _get_digest() method
  

• 3b881efc
  by Jürg Billeter at 2019-02-18T10:29:25Z

  element.py: Expand scope of timed_activity in _cache_artifact()
  
  ArtifactCache.commit() is no longer the only expensive part and even
  more so with the upcoming change to use CASBasedDirectory.
  

• 3832c0d1
  by Jim MacArthur at 2019-02-18T10:29:25Z

  element.py: Alter _cache_artifact to use CasBasedDirectory
  

• ef85e3b2
  by Jürg Billeter at 2019-02-18T11:18:10Z

  Merge branch 'jmac/cache_artifacts_with_vdir' into 'master'
  
  Cache artifacts with virtual directories instead of filesystem.
  
  Closes #787
  
  See merge request BuildStream/buildstream!991

• e0f0a01d
  by Chandan Singh at 2019-02-18T12:41:31Z

  requirements/Makefile: exclude pkg-resources
  
  Having `pkg-resources` in requirements files is never the right thing as
  it is not really a package that one can install. This is basically an
  artifact of Debian-based systems providing incorrect metadata to pip.
  See [this pip issue](https://github.com/pypa/pip/issues/4022) and
  [this Ubuntu issue](https://bugs.launchpad.net/ubuntu/+source/python-pip/+bug/1635463)
  for more background.
  

• f149fed2
  by Tristan Van Berkom at 2019-02-18T13:43:28Z

  Merge branch 'chandan/remove-pkg-resources' into 'master'
  
  requirements/Makefile: exclude pkg-resources
  
  Closes #918
  
  See merge request BuildStream/buildstream!1156

• 4022234e
  by Tristan Van Berkom at 2019-02-18T13:44:29Z

  .gitlab-ci.yml: Some new test environments were added, but not collecting coverage.
  
  The test jobs which create coverage reports need to be listed
  in the dependencies of the coverage job, as highlighted in
  issue #356.
  

• 86466e7e
  by Tristan Van Berkom at 2019-02-18T14:27:08Z

  Merge branch 'tristan/collect-missing-coverage' into 'master'
  
  .gitlab-ci.yml: Some new test environments were added, but not collecting coverage.
  
  See merge request BuildStream/buildstream!1157

• 2a4e4843
  by Chandan Singh at 2019-02-18T17:57:48Z

  loader: Allow dependencies to use ":" to refer to junctioned elements
  
  This will allow cross-junction dependencies to be listed as strings on a
  single line.
  
  As part of this, some logic around initializing `Dependency()` objects
  have been moved out of `_extract_depends_from_node()` method into the
  constructor of `Dependency()` class, to keep all related code in one
  place.
  
  * _loader/types.py: While initializing `Dependency` objects, attempt to
                      split filenames, only if no `junction` was specified
                      explicitly. If a `junction` was specified, then
                      filenames with `:` in their names will result in an
                      error.
  
  * _loader/loadelement.py: Refactor logic to initialize `Dependency()`
                            objects to move it to the `Dependency()`
                            constructor.
  
  * tests/frontend/buildcheckout.py: Add tests to ensure the above.
  
  * _versions.py: Bump BST_FORMAT_VERSION.
  
  Fixes #809.
  

• 33b2a79e
  by Chandan Singh at 2019-02-18T17:57:48Z

  NEWS: Cross-junction dependencies can now be specified inline
  

• 1124d1d8
  by Chandan Singh at 2019-02-18T17:57:48Z

  Document how to refer to cross-junction dependencies inline
  
  In the previous commit, we added support to express cross-junction
  dependencies inline as simple strings. Document it along with the
  version in which the feature was added.
  

30 changed files:

• .gitlab-ci.yml
• CONTRIBUTING.rst
• NEWS
• README.rst
• buildstream/_artifactcache.py
• + buildstream/_artifactelement.py
• buildstream/_cachekey.py
• buildstream/_cas/cascache.py
• buildstream/_context.py
• buildstream/_exceptions.py
• buildstream/_frontend/app.py
• buildstream/_frontend/cli.py
• buildstream/_frontend/widget.py
• buildstream/_loader/loadelement.py
• buildstream/_loader/loader.py
• buildstream/_loader/metaelement.py
• buildstream/_loader/types.py
• buildstream/_profile.py
• buildstream/_project.py
• buildstream/_stream.py
• buildstream/_variables.py
• buildstream/_versions.py
• buildstream/_yaml.py
• buildstream/data/projectconfig.yaml
• buildstream/data/userconfig.yaml
• buildstream/element.py
• buildstream/plugins/elements/junction.py
• buildstream/plugins/sources/tar.py
• buildstream/source.py
• buildstream/storage/_casbaseddirectory.py

The diff was not included because it is too large.

—
View it on GitLab.
You're receiving this email because of your account on gitlab.com. If you'd like to receive fewer emails, you can adjust your notification settings.