-
7285e87e
by William Salmon
at 2019-02-07T09:42:36Z
Add more log formatting options
The 'wallclock-us' is a option that we thought we already had.
-
e61f4713
by Will Salmon
at 2019-02-07T10:58:54Z
Merge branch 'willsalmon/log_formating' into 'master'
Add more log formatting options
See merge request BuildStream/buildstream!1125
-
b4d4c4f5
by Phil Dawson
at 2019-02-08T14:27:54Z
Expose basic api for testing external plugins.
We want external plugins to be able to make use of the core testing utils.
This commit exposes the basic utilities which are currently in use in
bst-external plugins. If necessary, more utilities could be exposed in the
future.
Moves the following files from tests/testutils/ to
buildstream/plugintestingutils/:
o runcli.py
o integration.py
As part of this, this commit makes the following changes to runcli.py
and integration.py:
o runcli.py: Fix linting errors
o runcli.py: Add user facing documentation
o Integration.py: Add user facing documentation
-
77ce0c16
by Phil Dawson
at 2019-02-08T16:28:05Z
Merge branch 'phil/plugin-testing-api' into 'master'
Expose basic api for testing external plugins.
Closes #847
See merge request BuildStream/buildstream!1075
-
1ee4a4ba
by Abderrahim Kitouni
at 2019-02-08T16:42:23Z
_artifactcache.py: don't leak the project specific remote caches
the code for initializing remotes added the project specific remote
caches to the global list instead of making a copy.
Fixes #618
-
24c0de16
by Abderrahim Kitouni
at 2019-02-08T16:42:23Z
_project.py: use artifact caches from the parent project for junctions
This makes a junction use the artifact cache of the parent project
before the ones defined for the junction
Fixes #401
-
a937f99a
by Javier Jardón
at 2019-02-08T20:29:05Z
Merge branch 'abderrahim/artifact-cache-junction' into 'master'
Use artifact cache specs from the parent project before those defined in junctions
Closes #618 and #401
See merge request BuildStream/buildstream!1113
-
da1560e4
by Javier Jardón
at 2019-02-09T09:10:01Z
.gitlab-ci.yml: Remove testing on Fedora 27
Fedora 27 is EOL since 2018-11-30
See https://fedoraproject.org/wiki/End_of_life
-
79fbab61
by Jürg Billeter
at 2019-02-09T10:30:20Z
Merge branch 'jjardon/fedora_27' into 'master'
.gitlab-ci.yml: Remove testing on Fedora 27
See merge request BuildStream/buildstream!1136
-
354c563d
by Jürg Billeter
at 2019-02-10T20:03:46Z
import.py: Validate config node
Fixes #662.
-
5e1be71f
by Jürg Billeter
at 2019-02-11T05:10:56Z
Merge branch 'juerg/import' into 'master'
import.py: Validate config node
Closes #662
See merge request BuildStream/buildstream!1141
-
99e1be45
by Jürg Billeter
at 2019-02-11T05:12:25Z
local.py: Do not follow symlinks in local directories
isdir() follows symlinks on the host, resulting in potential host
contamination. This change reorders the file checks to avoid this issue.
-
f95e222e
by Jürg Billeter
at 2019-02-11T05:12:25Z
sandbox/sandbox.py: Do not follow symlinks in _has_command()
This is required to ensure symlinks are not resolved on the host.
-
89973fb3
by Jürg Billeter
at 2019-02-11T05:12:25Z
utils.py: Remove list_dirs parameter from list_relative_paths()
list_dirs was always True in the BuildStream code base. There was also a
bug in the list_dirs=False code path as it did not return symlinks in
`dirnames`.
This is an API break, however, there are no known external callers.
-
d1da3fb0
by Jürg Billeter
at 2019-02-11T05:12:25Z
utils.py: Fix sorting of symlinks to directories
os.walk() resolves symlinks to check whether they point to a directory
even when followlinks is set to False. We already work around that
broken behavior by extracting symlinks from `dirnames`. However, the
sort order was still incorrect as we returned symlinks in dirnames
before files and other symlinks. This change fixes this, sorting all
files and symlinks in a single list.
-
7cf67ed3
by Jürg Billeter
at 2019-02-11T05:12:25Z
_casbaseddirectory.py: Do not mimic os.walk() in list_relative_paths()
This matches the change in utils.list_relative_paths() that now sorts
all symlinks as files, instead of following the broken behavior of
os.walk().
-
7ec0bb5e
by Jürg Billeter
at 2019-02-11T05:44:20Z
tests/sources/local.py: Add directory symlink test
-
c07cc967
by Jürg Billeter
at 2019-02-11T07:13:28Z
Merge branch 'juerg/symlinks' into 'master'
Symlink fixes
See merge request BuildStream/buildstream!1138
-
9e3d3e05
by Angelos Evripiotis
at 2019-02-11T07:14:50Z
contributing: snakeviz replaces pyflame+flamegraph
Replace the instructions for pyflame+flamegraph with simpler ones for
snakeviz. For our general use-case this seems to be easier and better.
Usage of this tool was demonstrated at the 2019 BuildStream Gathering in
January by Daniel Silverstone, when presenting the aggregate results of
profiling on many target environments.
Here is the relevant mailing list thread:
"Profiling before the gathering"
https://mail.gnome.org/archives/buildstream-list/2019-January/msg00057.html
-
bb6a692d
by Jürg Billeter
at 2019-02-11T09:20:34Z
Merge branch 'snakeviz' into 'master'
contributing: snakeviz replaces pyflame+flamegraph
See merge request BuildStream/buildstream!1129
-
4bbeb431
by Daniel Silverstone
at 2019-02-11T13:14:12Z
_yaml.py: Only retrieve provenance in node_get() when needed
We were indiscriminately retrieving the node's provenance data in the
`node_get()` function which was accounting for approximately a third of
the total runtime of `node_get()` which dominates pre-scheduler time in
`bst build`. This change ameliorates that situation by only retrieving
the provenance data when it's actually needed.
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
e59cbed0
by Daniel Silverstone
at 2019-02-11T13:14:13Z
_yaml.py: Reduce use of `isinstance()` in `node_sanitize()`
We know that nodes are typically one of:
str, list, dict, bool, tuple, NoneType or our ChainMap
Of these, dict and ChainMap are Mapping, only list is list
and the rest are returned unchanged. We can reduce/defer our use
of isinstance here, dramatically, improving performance.
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
69c7e263
by Daniel Silverstone
at 2019-02-11T13:14:13Z
_cachekey.py: Add a variant of the calculator which take pre-sanitized input
In order to support a case where the caller already has pre-sanitized input
this variant of the cache key generator needs to exist.
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
731fd503
by Daniel Silverstone
at 2019-02-11T13:14:13Z
element.py: Alter Element.__calculate_cache_key() to pre-santize inputs
In order to reduce the effort we spend in `_yaml.node_sanitize()` take
advantage of how `OrderedDict` works and pre-sanitize the majority of
the cache-key once. This approximately halves the amount of effort we
spend in `_yaml.node_sanitize()` in pre-scheduler build scenarios.
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
b6d128af
by Daniel Silverstone
at 2019-02-11T13:58:22Z
_yaml.py: Remove use of isinstance() in `{node,list}_{chain_,}copy`
A non-trivial proportion of time pre-scheduler in `bst build` is spent
copying (or chain-copying) nodes. Approximately a quarter of the time
spent in that effort is in `isinstance()`. This removes that CPU load.
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
