-
3ca487b8
by Chandan Singh
at 2018-10-08T18:25:39Z
bst-docker-import: Consistently use stderr for all logs
In !857, we added `contrib/bst-docker-import` script but it has a small
issue that some of logs go to stdout while others go to stderr. Fix it
so that all logging is done on stderr.
-
49df3d75
by Chandan Singh
at 2018-10-08T18:58:54Z
Merge branch 'chandan/fix-bst-docker-import-logging' into 'master'
bst-docker-import: Consistently use stderr for all logs
See merge request BuildStream/buildstream!863
-
d827dfae
by Angelos Evripiotis
at 2018-10-09T09:12:45Z
contributing.rst: end lines with punctuation
-
9590e8ae
by Angelos Evripiotis
at 2018-10-09T09:12:45Z
contributing.rst: fix whitespace
No tabs, no lines of only horizontal whitespace.
-
6d02e269
by Angelos Evripiotis
at 2018-10-09T09:12:45Z
contributing.rst: fix typo of 'get_count'
-
3ed26a47
by Angelos Evripiotis
at 2018-10-09T09:12:45Z
contributing.rst: de-paren subclass example
It seems easier to read when separated into two sentences.
-
5b22d850
by Angelos Evripiotis
at 2018-10-09T09:12:45Z
contributing.rst: no spaces before '?'
-
ac0776f8
by Tristan Van Berkom
at 2018-10-09T09:36:33Z
Merge branch 'aevri/contributing_fixups' into 'master'
Minor fixups to contributing.rst
See merge request BuildStream/buildstream!866
-
885bd946
by Tristan Van Berkom
at 2018-10-09T09:39:24Z
CONTRIBUTING.rst: Added note about sphinx supporting docstrings on instance variables
Python does not natively support this, but sphinx does parse them and includes
these in the generated documentation.
-
120d8c73
by Tristan Van Berkom
at 2018-10-09T09:45:23Z
CONTRIBUTING.rst: Fix typos in previous commit.
Ooops, that was done far too quickly.
-
cda1354c
by Chandan Singh
at 2018-10-10T00:17:51Z
doc: Add new document about Docker integrations
Add a new section under "Additional writings" section about how
BuildStream integrates with Docker. At present, this section includes
only two sections:
- "Run BuildStream inside Docker", that refers to existing instructions
for doing so
- "Generate Docker images", that adds instructions for using the newly
added `contrib/bst-docker-import` script
In future, this document can be extended to include the following
things:
- Importing Docker sources via `docker` source plugin - if/when the
Docker source plguin is moved from bst-external to BuildStream proper
- A Docker sandbox, if/when that is added
-
dd472d95
by Chandan Singh
at 2018-10-10T00:18:17Z
NEWS: Add entry about contrib/bst-docker-import
As suggested in
https://gitlab.com/BuildStream/buildstream/merge_requests/857#note_106809743,
add a NEWS entry for `contrib/bst-docker-import` that was added
recently.
-
35ac26a7
by Chandan Singh
at 2018-10-10T00:59:17Z
Merge branch 'chandan/bst-and-docker' into 'master'
Add documentation and NEWS entry about bst-docker-import
See merge request BuildStream/buildstream!864
-
7d96333f
by Laurence Urhegyi
at 2018-10-10T19:24:14Z
README: Update to add link to website
-
39492db8
by devcurmudgeon
at 2018-10-10T19:24:14Z
Merge branch 'patch-2' into 'master'
README: Update to add link to website
See merge request BuildStream/buildstream!859
-
4f0bfb4a
by Tristan Van Berkom
at 2018-10-11T15:37:51Z
Rename element_enums.py -> types.py
This will be the place to store low level data types used
throughout the core, for now this includes public and private
types.
-
9dc10cc7
by Tristan Van Berkom
at 2018-10-11T15:45:36Z
doc/source/core_framework.rst: Include the foundation types in the API docs.
-
efc0d4cc
by Tristan Van Berkom
at 2018-10-11T15:48:44Z
types.py: Moved Consistency definition here from source.py
And slightly touch up the doc strings for Consistency and Scope.
-
a0712ead
by Tristan Van Berkom
at 2018-10-11T16:23:54Z
Merge branch 'tristan/refactor-types-api' into 'master'
Refactor types api
See merge request BuildStream/buildstream!870
-
ce55b9a0
by Tiago Gomes
at 2018-10-12T10:15:46Z
doc: updates considering website being live now
* Add a link to the website on the main page.
* Remove install instructions as they are now on the website.
* Remove Resources section as that information can be found at the
website, and also looks bad.
* Move artifact server setup from the no longer existing Install section
to the Using section.
-
10b092e1
by Tristan Van Berkom
at 2018-10-12T10:21:27Z
doc/source/additional_docker.rst: Fix link to refer to website
The docker guide, which is part of the install guide, has moved
to the website.
-
1c7ba9a0
by Tristan Van Berkom
at 2018-10-12T10:24:31Z
doc/source/index.rst: Moved references to the website below the simple ToC.
-
3738dd06
by Tristan Van Berkom
at 2018-10-12T10:49:39Z
Merge branch 'tristan/remove-install-guide' into 'master'
Remove install guide
See merge request BuildStream/buildstream!872
-
c31ed138
by Tristan Van Berkom
at 2018-10-14T15:24:53Z
CONTRIBUTING.rst: Added more guidelines about documenting the user guide
-
b39d87b4
by Jürg Billeter
at 2018-10-15T11:07:05Z
_artifactcache: Fix crash in push_directory() without artifact server
Fixes #708.
-
4e3ec89e
by Jürg Billeter
at 2018-10-15T11:48:16Z
Merge branch 'juerg/remote-config' into 'master'
_artifactcache: Fix crash in push_directory() without artifact server
Closes #708
See merge request BuildStream/buildstream!871
-
d15ed8fc
by Jonathan Maw
at 2018-10-15T15:54:59Z
yaml: Add a cache of parsed and provenanced yaml
Note that the ProvenanceFile's names will be incorrect after loading
from the cache, but this is currently only used for writeback, which
isn't used in junctions.
-
f4eb685a
by Jonathan Maw
at 2018-10-15T15:54:59Z
yamlcache.py: Add yaml cache tests
-
26dfa882
by Lachlan Mackenzie
at 2018-10-15T15:54:59Z
yaml.py: Add loader for yaml test files cache option
* Fix to CWD issue provided by Jonathan Maw
-
0fbff08b
by Lachlan Mackenzie
at 2018-10-15T15:54:59Z
yaml.py: Add yaml cache testing to yaml list composition test
-
3d58d7c5
by Lachlan Mackenzie
at 2018-10-15T15:54:59Z
yaml.py: Add yaml cache test to yaml list composition twice
-
363cfc39
by Lachlan Mackenzie
at 2018-10-15T15:54:59Z
yamlcache.py: Add YAML cache changed file test
* Test in same style as test_yamlcache_used
* Move of project not required so removed
-
086c47c4
by Jonathan Maw
at 2018-10-15T16:19:01Z
Merge branch 'lachlan/pickle-yaml-test-list-composite' into 'master'
Add YAM cache testing to yaml tests and further YAML cache refinements
See merge request BuildStream/buildstream!836
-
6298e409
by William Salmon
at 2018-10-16T12:39:14Z
Sandbox: CWD was not being created for workspaces
The code was creating the cwd folder but when the workspace was
mounted in to the buildroot it was hiding the folder created in it
behind the bind mounted workspace.
However by using the bubblewarp `--dir` directive to ensure that cwd
exists we can cover both workspace and non workspace situations with
the same method.
For issue #512 in Gitlab.
-
bbcb9bd5
by William Salmon
at 2018-10-16T12:39:14Z
New Integration test for workspace command-subdir
Build stream was only creating command-subdir for non workspace builds
this has now been fixed, and this regression test should support the
feature.
For issue #512 in Gitlab.
-
5e7cc645
by William Salmon
at 2018-10-16T12:39:14Z
Add conf-root variable to builds
Adding the conf-root variable makes creating out of source builds
slightly easier.
For issue #512 in Gitlab.
-
0d04e1b7
by William Salmon
at 2018-10-16T12:39:14Z
Added tests for conf-root variable
For issue #512 in Gitlab.
-
b2767ac0
by William Salmon
at 2018-10-16T12:39:14Z
Extended the Documentation to cover out of source builds
For issue #512 in Gitlab.
-
118704a9
by William Salmon
at 2018-10-16T12:39:14Z
Added a news item for building out of the source directory
For issue #512 in Gitlab.
-
0a5db229
by Phil Dawson
at 2018-10-16T13:19:56Z
Merge branch 'willsalmon/outOfSourecBuild' into 'master'
Out of source builds
Closes #512
See merge request BuildStream/buildstream!776
-
1fd38fb7
by Jim MacArthur
at 2018-10-16T14:54:09Z
bst2html.py: Name the second argument to avoid an error
-
b023d5c0
by Jim MacArthur
at 2018-10-16T15:28:15Z
Merge branch 'jmac/doc-argument-fix' into 'master'
Add named argument in bst2html.py to fix documentation
See merge request BuildStream/buildstream!874
-
ac5b482d
by Richard Dale
at 2018-10-16T19:40:39Z
Fix spelling of it's and its
-
3d9c4eca
by Jürg Billeter
at 2018-10-16T20:17:39Z
Merge branch 'rdale/spelling' into 'master'
Fix spelling of it's and its
See merge request BuildStream/buildstream!877
-
edc07536
by Tiago Gomes
at 2018-10-17T13:49:15Z
README: use correct links for documentation website
-
155296a7
by Tiago Gomes
at 2018-10-17T13:49:15Z
setup.py: use correct link for documentation website
-
131f95eb
by Tiago Gomes
at 2018-10-17T14:15:38Z
Merge branch 'tiagogones/update-links-to-documentation' into 'master'
Update links to documentation website
See merge request BuildStream/buildstream!876
-
c327d1f3
by Valentin David
at 2018-10-17T14:33:40Z
Change URL to the Alpine tarball
-
eeb334f2
by Javier Jardón
at 2018-10-17T15:03:17Z
Merge branch 'valentindavid/move_alpine_tarball' into 'master'
Change URL to the Alpine tarball
See merge request BuildStream/buildstream!880
-
b8f69946
by Daniel Silverstone
at 2018-10-18T09:32:09Z
_loader/loader.py: Be more explanatory in CircDep exception
When a circular dependency is detected, the reported error hilights
only one part of the loop in question. This change ensures that
the entire loop is listed so that the user has a greater chance of
easily determining the correct fix.
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
892866e1
by richardmaw-codethink
at 2018-10-18T10:04:32Z
Merge branch 'danielsilverstone-ct/report-loop' into 'master'
_loader/loader.py: Be more explanatory in CircDep exception
See merge request BuildStream/buildstream!878
-
f1aa0974
by Javier Jardón
at 2018-10-18T14:13:35Z
Fix python warnings: Use collections.abc instead collections
Since python 3.3, collections has been moved to collections.abc module.
For backwards compatibility, they continue to be visible in this module through Python 3.7.
Subsequently, they will be removed entirely.
See https://docs.python.org/3/library/collections.html
-
f1550c8e
by Valentin David
at 2018-10-18T16:02:45Z
Merge branch 'jjardon/warnings_collections' into 'master'
Fix python warnings: Use collections.abc instead collections
See merge request BuildStream/buildstream!883
-
1c24979a
by Angelos Evripiotis
at 2018-10-19T21:17:01Z
Spelling fixes: 'affect', not 'effect' as verb
Fix most instances of 'effect' being used as a verb instead of a noun.
Skipped generated instance in doc/source/conf.py.
-
43a2eee3
by Angelos Evripiotis
at 2018-10-19T21:48:59Z
Merge branch 'aevri/affect' into 'master'
Spelling fixes: 'affect', not 'effect' as verb
See merge request BuildStream/buildstream!885
-
12719f0d
by Jürg Billeter
at 2018-10-22T17:05:41Z
_artifactcache: Rename update_atime() to update_mtime()
os.utime() updates both, however, list_artifacts() sorts refs by mtime,
i.e., atime is irrelevant.
-
a7a28d14
by Jürg Billeter
at 2018-10-22T17:05:41Z
_artifactcache: Fix ref in update_mtime()
get_artifact_fullname() is required to construct the ref. The cache key
alone does not suffice.
-
be29e0f5
by Jürg Billeter
at 2018-10-22T17:53:26Z
Merge branch 'juerg/cas-mtime' into 'master'
_artifactcache: Fix ref in update_mtime()
See merge request BuildStream/buildstream!882
-
b74aca1a
by Jürg Billeter
at 2018-10-23T09:22:19Z
tests/testutils/artifactshare.py: Fix has_artifact() to match core code
Replace path separator with '-' to match Element.normal_name.
-
c7dda150
by Jürg Billeter
at 2018-10-23T09:48:00Z
Merge branch 'juerg/test-artifactshare' into 'master'
tests/testutils/artifactshare.py: Fix has_artifact() to match core code
See merge request BuildStream/buildstream!890
-
74c115b9
by Angelos Evripiotis
at 2018-10-23T10:07:31Z
plugins/sources: refactor, rm needless ExitStack-s
Remove two uses of ExitStack that aren't necessary and make the code
a little harder to follow.
-
ecb58b42
by Phil Dawson
at 2018-10-23T10:33:47Z
Merge branch 'aevri/rm-exitstack' into 'master'
plugins/sources: refactor, rm needless ExitStack-s
See merge request BuildStream/buildstream!888
-
aa0cbf5d
by Martin Blanchard
at 2018-10-23T10:54:40Z
_sandboxremote.py: Try to reopen operation steam on failure
The REAPI allows a client to reconnect to an ongoing operation stream by
providing a WaitExecution(). If implemented on server side, BuildStream
will try to recover from connection errors using it.
https://gitlab.com/BuildStream/buildstream/issues/630
-
552f5fc6
by Jim MacArthur
at 2018-10-23T11:19:48Z
Merge branch 'mablanch/630-remote-execution-reconn' into 'master'
Handle connection losses during remote build execution
See merge request BuildStream/buildstream!806
-
276d2195
by Angelos Evripiotis
at 2018-10-25T10:17:49Z
_downloadablefilesource: handle ValueError-s
-
2d012a21
by Angelos Evripiotis
at 2018-10-25T10:17:49Z
Merge branch 'aevri/unknown-uri-452' into 'master'
_downloadablefilesource: handle ValueError-s
Closes #452
See merge request BuildStream/buildstream!893
-
cf4741ce
by James Ennis
at 2018-10-25T13:05:43Z
docs: Add helpful notes to artifact server configuration instructions
-
5737bb06
by James Ennis
at 2018-10-25T13:05:43Z
using_configuring_artifact_server.rst: Fix incorrect reference to bst push (with link)
-
28317622
by James Ennis
at 2018-10-25T13:35:22Z
Merge branch 'jennis/patch_remote_cache_docs' into 'master'
Improve our documentation for configuring an artifact server
See merge request BuildStream/buildstream!899
-
d2743c52
by Daniel Silverstone
at 2018-10-25T14:43:52Z
_context.py: Remove useless `return` from `Context.message`
This pointless bare `return` was causing modern pylint to raise an
error.
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
95adf09e
by Daniel Silverstone
at 2018-10-25T14:43:52Z
_variables.py: Fix lint issue in `find_recursive_variable`
The python `for`/`else` construct only makes sense if the `for` loop body
has a `break` statement. Otherwise the use of `else` causes pylint
to raise an error.
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
9359c067
by Daniel Silverstone
at 2018-10-25T14:43:52Z
_workspaces.py: Simplify chained comparison
In `_parse_workspace_config` there was a chained version comparison
which is simplified in this commit to quieten a pylint issue.
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
796bd8c9
by Daniel Silverstone
at 2018-10-25T14:43:52Z
_yaml.py: use `in (a,b)` to simply boolean checks
Where we use a construct `val == foo or val == bar` we can instead use
`val in (foo, bar)` which pylint prefers.
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
5a3a537f
by Daniel Silverstone
at 2018-10-25T14:43:52Z
_yaml.py: Quieten a lint in Python < 3.6
For Python before 3.6, `path.resolve()` could not take the `strict`
keyword argument. Linting on such Python versions will raise an unnecessary
issue given the check present. As such, quieten that lint.
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
b279c730
by Daniel Silverstone
at 2018-10-25T14:43:52Z
_yamlcache.py: Correct bug in `YamlCache._get_filepath()`
The correct file path was computed but never returned. This was
never caught because of a test missing consumption of one of its
parameters. This commit resolves the bug mentioned in #722 leaving
the test to be amended by a future commit.
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
527f3b32
by Daniel Silverstone
at 2018-10-25T14:43:52Z
_yamlcache.py: Correct braino in variable name
This braino would mean that we'd fail to raise the right exception
if we failed to find the right project object when unpickling the
yaml cache.
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
6d0c697d
by Daniel Silverstone
at 2018-10-25T14:43:52Z
_yamlcache.py: Remove unused imports
The imports of _cachekey and utils were unused.
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
32323695
by Daniel Silverstone
at 2018-10-25T14:43:52Z
buildelement.py: Quieten a lint about `__commands`
Unfortunately elements don't override `__init__` in general, so
we have to quieten pylint here.
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
c5c0654d
by Daniel Silverstone
at 2018-10-25T14:43:52Z
element.py: Simplify some conditions with `in (foo, bar)`
Where we have conditions of the form `var == foo or var == bar` it
can be simplified to `var in (foo, bar)` which pylint prefers.
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
4f84e2f6
by Daniel Silverstone
at 2018-10-25T14:43:52Z
element.py: Silence pylint warning about abstract method
Unfortunately pylint can't understand that we'll always use subclasses
of Element and as such complains of a no-return function (`assemble()`)
having its return value used. This quietens that warning.
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
b4645745
by Daniel Silverstone
at 2018-10-25T14:43:52Z
plugin.py: use dict.get() instead of an if statement
Pylint recommends the use of dict.get() rather than a multi-line if
statement for handling optional keyword arguments.
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
68ec9bcf
by Daniel Silverstone
at 2018-10-25T14:43:52Z
source.py: Quiet several pylint issues with abstract methods
A number of times, abstract methods are called which pylint cannot
possibly understand will only ever happen in subclasses which implement
the abstract methods. This silences those specific warnings
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
1bf0a031
by Daniel Silverstone
at 2018-10-25T14:43:52Z
source.py: Remove unused local variable
Remove `source_kind` as it was unused.
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
e9c6db1e
by Daniel Silverstone
at 2018-10-25T14:43:52Z
source.py: Remove unused variable
Remove the `context` variable which was unused.
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
5a187a1e
by Daniel Silverstone
at 2018-10-25T14:43:52Z
utils.py: Disable lint warning about popen
We use `preexec_fn` in our use of Popen. Since there are threads
in the codebase, this is not necessarily safe. However we go to
great lengths to ensure that the main process doesn't spawn threads
(they're used in gRPC a lot) and so it should be safe for our use.
As such, we disable the lint here.
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
b368f57f
by Daniel Silverstone
at 2018-10-25T14:43:52Z
utils.py: Group stat imports
The imports of `stat` and `S_ISDIR` from `stat` should be grouped to
satisfy pylint.
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
b45fea19
by Daniel Silverstone
at 2018-10-25T14:43:52Z
_artifactcache/artifactcache.py: Remove superfluous parens
The pylint report dislikes the superfluous parens which used to
be here in this assert.
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
3e4a6507
by Daniel Silverstone
at 2018-10-25T14:43:52Z
_artifactcache/artifactcache.py: Silence lint issues
The pylint tool is unable to understand that the abstract methods
in the artifact cache will only be called when there's concrete
implementations behind them. Silence the lint errors for these
specific calls.
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
447780b0
by Daniel Silverstone
at 2018-10-25T14:43:52Z
_frontend/app.py: Remove unused import of `resource`
The `resource` import was unused. Removed.
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
247b9fa8
by Daniel Silverstone
at 2018-10-25T14:43:52Z
_frontend/status.py: Reorder imports
The `curses` import is "standard" whereas `click` is not. As such, despite
alphabetical order normally being the rule, `curses` must come before `click`.
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
522867c8
by Daniel Silverstone
at 2018-10-25T14:43:52Z
_fuse/{hardlinks,mount}.py: Remove dangerous use of {} in defaults
Since default values in arguments to functions and methods are created once
at the compilation of the code, it is dangerous to include list and dict
literals in them. This changes the use of {} to None.
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
31a7bce5
by Daniel Silverstone
at 2018-10-25T14:43:52Z
_fuse/mount.py: Deal with abstract method lint issue
Unfortunately pylint can't tell this is an abstract method which will
only ever end up invoked from a class which concretises it. As such
we need to silence this warning.
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
eb805c6c
by Daniel Silverstone
at 2018-10-25T14:43:52Z
_options/optionbool.py: Refactor conditions
The pylint tool prefers that conditions of the form "var == foo or var == bar"
be refactored into "var in (foo, bar)".
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
aca9a612
by Daniel Silverstone
at 2018-10-25T14:43:52Z
_platform/darwin.py: Remove useless __init__()
An otherwise empty __init__ passing no arguments to super.__init__
is worthless. Remove it.
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
77b1506f
by Daniel Silverstone
at 2018-10-25T14:43:52Z
_platform/darwin.py: Remove unused imports
The import of `resource` was unused.
The import of `PlatformError` was unused.
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
70000b0d
by Daniel Silverstone
at 2018-10-25T14:43:52Z
_platform/linux.py: Remove unused imports
The imports of `Message` and `MessageType` were unused.
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
bab617d0
by Daniel Silverstone
at 2018-10-25T14:43:52Z
_platform/linux.py: Simplify return statement
The linter dislikes an if/return/else/return where it can be
simplified to a return of the if condition.
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
45a1d15f
by Daniel Silverstone
at 2018-10-25T14:43:52Z
_scheduler/jobs/job.py: Correct lint issue around abstract method
The linter cannot detect abstract methods the way we use them, so
this silences the warning which ensues.
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
e9154b4d
by Daniel Silverstone
at 2018-10-25T14:43:52Z
_scheduler/queue/pullqueue.py: Remove unused return value
The `done()` method does not return anything anywhere else.
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
7b082bb7
by Daniel Silverstone
at 2018-10-25T14:43:52Z
_scheduler/queue/trackqueue.py: Remove unused import
The `SourceError` import was never used
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
f1cfc0b7
by Daniel Silverstone
at 2018-10-25T14:43:52Z
plugins/source/git.py: Rationalised unused return values
The `assert_ref_in_track()` method's return value was never used so
rationalise it to never return a value.
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
7c72d25b
by Daniel Silverstone
at 2018-10-25T14:43:52Z
sandbox/_mount.py: Do not use dict literals in argument defaults
The use of dictionary literals in argument defaults is disrecommended
due to the way that they are static and thus potentially very confusing.
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
4e7408ee
by Daniel Silverstone
at 2018-10-25T14:43:52Z
sandbox/_mounter.py: Remove useless inheritance on `object`
In Python 3, inheriting from `object` explicitly is pointless.
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
df495d16
by Daniel Silverstone
at 2018-10-25T14:43:52Z
sandbox/_sandboxbwrap.py: Disable lint about dict.get
Sometimes `dict.get()` is preferable, in this instance it's less clear
so we'll disable that lint
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
63c6e6b1
by Daniel Silverstone
at 2018-10-25T14:43:52Z
sandbox/_sandboxchroot.py: Silence warning about unused variable
The variable was unused, remove it.
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
3394741a
by Daniel Silverstone
at 2018-10-25T14:43:52Z
sandbox/_sandboxchroot.py: Silence warning about preexec_fn
We are super-careful to not use threads in places where we might use
Popen and as such this warning is save to quash.
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
cd20e7a1
by Daniel Silverstone
at 2018-10-25T14:43:52Z
sandbox/_sandboxchroot.py: Silence warning about dict.get
Sometimes `dict.get()` is preferable to an `if` statement, but this time
it's clearer if we keep the structure. As such, silence the warning.
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
5087f5c5
by Daniel Silverstone
at 2018-10-25T14:43:52Z
_artifactcache/cascache.py: Refactor slightly for indentation
Correct some multi-line conditional indentation by slight refactor
if necessary.
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
d73d965b
by Daniel Silverstone
at 2018-10-25T14:43:52Z
_artifactcache/cascache.py: Silence len(SEQUENCE) as condition warnings
The lint tool dislikes the use of `if len(SEQ) == 0` type statements
because nominally `len(SEQ)` may be more expensive than `bool(SEQ)`.
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
fa740402
by Daniel Silverstone
at 2018-10-25T14:43:52Z
_artifactcache/cascache.py: Prepare attributes in __init__
Python linters prefer that attributes are defined in __init__ rather
than left to later routines.
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
ad8271f0
by Daniel Silverstone
at 2018-10-25T14:43:52Z
_artifactcache/cascache.py: Remove unused imports
The imports of `Message` and `MessageType` were unused.
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
1445a901
by Daniel Silverstone
at 2018-10-25T14:43:52Z
_artifactcache/cascache.py: Cleanup lint warning about indexing
In order to quieten the linter, and to make a more correct comparison,
we switch to using the `Code.OK` enumeration from `google.rpc.Code`
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
19df07ea
by Tristan Maat
at 2018-10-25T14:43:52Z
Bump CI image tags
We need to do this so that the new linting errors pop up
-
64dbe9ae
by Daniel Silverstone
at 2018-10-25T14:43:52Z
MANIFEST: Include a lot more missing stuff in the manifest
Versioneer needs to be in the MANIFEST.in and its .pyc needs to be ignored
in the .gitignore. Also much docs were not being included, nor conftest.py.
Much of the test suite data files needed including, so changed that to a
raw include of everything in the tests/ tree.
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
f6bc25dd
by Daniel Silverstone
at 2018-10-25T14:43:52Z
.gitlab-ci.yml: Change how coverage is recovered
We need to alter how coverage data is recovered in order to work with
newer coverage tooling.
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
1c1d14e7
by Chandan Singh
at 2018-10-25T15:04:00Z
Merge branch 'danielsilverstone-ct/fix-lint-issues' into 'master'
Fix lint issues
See merge request BuildStream/buildstream!892
-
23140b6b
by Chandan Singh
at 2018-10-25T15:25:22Z
sources: Don't redundantly document directory configuration
Instead of redundantly documenting "directory" configuration option in
each source plugin, refer to the built-in functionality documentation of
the Source class.
* source.py: Add anchor for built-in functionality.
* sources/plugins/*.py: Remove directory config option, and add reference to
Source's built-in functionality.
Fixes #713.
-
db2a676e
by Chandan Singh
at 2018-10-25T15:26:39Z
build elements: Add reference to BuildElement's built-in config
For each element plugin derived from BuildElement, add reference to the
built-in functionality documentation of BuildElement.
* buildelement.py: Add anchor for built-in functionality.
* plugins/elements/*.py: Add refence to BuildElement's built-in
functionality for each build element.
Related to #713.
-
356d84cd
by Chandan Singh
at 2018-10-25T16:59:13Z
Merge branch 'chandan/source-doc-directory' into 'master'
plugins: Don't redundantly document common configurations
Closes #713
See merge request BuildStream/buildstream!896
-
8c41b5b8
by Daniel Silverstone
at 2018-10-26T09:02:30Z
_yaml.py: Do not insert into cache if retrieved from cache
We were unconditionally inserting into the yaml cache anything we had
retrieved from the yaml cache. This had the side-effect of being expensive
on re-runs of commands where yaml hadn't changed.
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
494d7018
by Jonathan Maw
at 2018-10-26T09:26:36Z
Merge branch 'danielsilverstone-ct/do-not-reinsert-yaml' into 'master'
_yaml.py: Do not insert into cache if retrieved from cache
See merge request BuildStream/buildstream!902
-
80762ecb
by Benjamin Schubert
at 2018-10-26T09:45:39Z
Check if command is a str and replace by list before checking existence
Currently we would make sure the sandbox had a command before
converting it to a list if it was given as a string. That meant that a string
command would never exist and the check be invalid.
This also adds the same logic in the dummy sandbox for consistency.
-
f131c407
by Benjamin Schubert
at 2018-10-26T10:09:51Z
Merge branch 'bschubert/fix-command-sandbox' into 'master'
Check is command is a str and replace by list before checking existence
Closes #728
See merge request BuildStream/buildstream!898
-
54d89aa1
by Daniel Silverstone
at 2018-10-26T10:20:36Z
_loader/loadelement.py: Use enumerate() in dependency extraction
In cases where elements end up with large numbers of dependencies,
the n-squared performance in `_extract_depends_from_node` can produce
significant delays during project load. This patch corrects that.
In addition we were double-retrieving provenance which was unnecessary.
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
e4c18ba4
by Phil Dawson
at 2018-10-26T10:43:55Z
Merge branch 'danielsilverstone-ct/use-enumerate' into 'master'
_loader/loadelement.py: Use enumerate() in dependency extraction
See merge request BuildStream/buildstream!901
-
20e5c58a
by James Ennis
at 2018-10-26T11:14:58Z
using_configuring_artifact_server.rst: Fix lines that are too long
-
320dae02
by James Ennis
at 2018-10-26T11:37:42Z
docs: Add more information to "managing the cache with systemmd" section
-
1eb993bf
by James Ennis
at 2018-10-26T12:02:49Z
Merge branch 'jennis/another_docs_patch' into 'master'
More information regarding the use of systemd to manage the cache
See merge request BuildStream/buildstream!904
-
f069d82f
by Daniel Silverstone
at 2018-10-26T12:19:58Z
various: Move _sentinel from utils.py to _yaml.py
The `_sentinel` in `utils.py` was used only for yaml node tracking.
As such, simplify matters by removing it from `utils.py` and move it
to `_yaml.py` which means that we no longer need to try and avoid a
circular import issue by means of runtime importing.
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
283887a5
by James Ennis
at 2018-10-26T13:12:12Z
Merge branch 'danielsilverstone-ct/yaml-sentinel-rework' into 'master'
Move _sentinel from utils.py to _yaml.py
See merge request BuildStream/buildstream!903
-
2c6cb230
by Valentin David
at 2018-10-26T14:21:18Z
Fix some ruamel warnings during tests
-
a8250ca4
by Valentin David
at 2018-10-26T15:04:25Z
Merge branch 'valentindavid/ruamel_warnings' into 'master'
Fix some ruamel warnings during tests
See merge request BuildStream/buildstream!889
-
1b308fe3
by Daniel Silverstone
at 2018-10-26T15:50:42Z
_stream.py: Preserve stdout FD across checkout-to-stdout
In order to support things which might need to use the stdout FD
after checkout has completed writing a tarball to it, preserve it
by means of `os.dup()` and `os.dup2()`
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
65d90c31
by Daniel Silverstone
at 2018-10-26T15:50:54Z
tests/testutils/runcli.py: Support binary-mode capture of stdout
In order to test things which write tarballs to stdout correctly,
we need to capture the binary output cleanly. This ensures we're not
potentially mismatching encodings in and out.
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
ed733f3e
by Daniel Silverstone
at 2018-10-26T15:51:40Z
tests/frontend/buildcheckout.py: Enable test_build_checkout_tarball_stdout
With the new support for saving the stdout FDs cleanly, and binary mode
capture in runcli, enable the tarball-to-stdout test.
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
e97d2a75
by Benjamin Schubert
at 2018-10-29T10:17:07Z
Merge branch 'bschubert/fix-binary-output-capture' into 'master'
Don't close stdout when running 'checkout' to stdout
Closes #486
See merge request BuildStream/buildstream!905
-
1625de0e
by Tristan Van Berkom
at 2018-10-29T11:49:33Z
doc/source/conf.py: Added `images` directory for statically included files
-
38507b4e
by Tristan Van Berkom
at 2018-10-29T11:49:33Z
doc/source/format_intro.rst: Adding link anchor for include directives
-
26164bcd
by Tristan Van Berkom
at 2018-10-29T13:57:51Z
MANIFEST.in: Include SVG and ODG files in source distributions
-
3a23c8f6
by Tristan Van Berkom
at 2018-10-29T13:57:51Z
doc: Adding new architecture document with initial "Overview of modules" section
-
41d370f6
by Tristan Van Berkom
at 2018-10-29T13:57:51Z
doc: Adding new architecture document describing the data model
-
bed7f7ac
by Tristan Van Berkom
at 2018-10-29T13:57:51Z
doc: Adding new architecture document describing the dependency model
-
9bdf7f40
by Tristan Van Berkom
at 2018-10-29T13:57:51Z
doc: Adding new architecture document about highlevel program flow
-
6b8bdf35
by Tristan Van Berkom
at 2018-10-29T13:57:51Z
doc: Adding new architecture document about how the scheduler works.
-
b1fadaa5
by Tristan Van Berkom
at 2018-10-29T13:57:51Z
doc: Moved cachekey and sandboxing writings into the architecture section
-
baf6b578
by Tristan Van Berkom
at 2018-10-29T14:28:41Z
Merge branch 'tristan/architecture-docs' into 'master'
Tristan/architecture docs
See merge request BuildStream/buildstream!907
-
09ef8b25
by Jonathan Maw
at 2018-10-30T15:42:05Z
element.py: Always clean up the rootdir
We shouldn't need it to persist now that we cache failed build dirs.
This change breaks the test
`tests/integration/shell.py::test_sysroot_workspace_visible`.
I can no longer see a use-case for this test.
AIUI, it tested that the failed build sysroot stored in the builddir has
the workspace's files in, despite the workspace being unmounted.
I believe this behaviour is made redundant by cached buildtrees.
This fixes part of #539
-
37afde0a
by Jonathan Maw
at 2018-10-30T15:42:05Z
Element: Use cached buildtree in build shells and failure shells
This includes changes in app.py:
* Interactive failure shell no longer uses the failed build sysroot,
defaulting to the cached build tree.
Changes in element.py are:
* Errors caused by building don't store the failed build sysroot,
instead storing that a sandbox can be created to debug the error.
* When staging sources, will stage the element's cached build tree if it
exists.
Changes in _exceptions.py:
* BstError.sandbox is now a flag of whether a sandbox can be opened up
to debug the error.
Changes in widget.py:
* Don't try to print any information about the sandbox.
Changes in _message.py:
* Fix documentation so Message.sandbox is not a directory any more.
This is part of #539
-
4bc71773
by Jonathan Maw
at 2018-10-30T15:42:05Z
NEWS: Add item for cached buildtree behaviour
-
9b89564f
by Jonathan Maw
at 2018-10-30T15:42:05Z
tests: Add test that cached build trees are staged in build shells
This is related to #539
-
ec76cbe1
by Jonathan Maw
at 2018-10-30T15:42:05Z
sandbox.py: Remove redundant Sandbox.__directory
-
825963b5
by Jonathan Maw
at 2018-10-30T15:42:05Z
element: Make "--sysroot" take a bare directory
i.e. instead of taking a directory that must contain "root" and
"scratch", and treating "root" as the root, use the directory directly.
In element.py:
* __sandbox takes the `bare_sandbox` arg, to pass into the sandbox's constructor
In sandbox.py:
* If bare_sandbox, `_root` is the passed-in directory, and `__scratch`
is None.
* Trying to use `__scratch` when bare_sandbox is True is a bug.
In _mount.py:
* Don't get the value of `__scratch` if it's not needed.
This is part of #539
-
cf721838
by Jonathan Maw
at 2018-10-30T15:42:05Z
NEWS: Add item for bst shell --sysroot changes
-
79d1bb7f
by Jonathan Maw
at 2018-10-30T15:42:05Z
tests: Add tests for 'bst shell --sysroot'
This is related to #539
-
c7ac7e7d
by Jonathan Maw
at 2018-10-30T16:19:17Z
Merge branch 'jonathan/debug-remote-failed-builds' into 'master'
Jonathan/debug remote failed builds
See merge request BuildStream/buildstream!869
-
516e990e
by ctolentino8
at 2018-10-31T11:36:46Z
_frontend/app.py: Set correct element-path in interactive bst-init
-
b8a37a63
by Tristan Van Berkom
at 2018-11-01T10:16:25Z
Merge branch 'chiaratolentino/bst-init-interactive-elementpath' into 'master'
_frontend/app.py: Set correct element-path in interactive bst-init
See merge request BuildStream/buildstream!912
-
b27b592a
by Benjamin Schubert
at 2018-11-01T10:49:57Z
Remove dependency on pytest-runner
This includes a new command mimicking pytest-runner so that we
can drop this dependency
This was the only setup_requires dependency that we had and
will make like easier for people behind proxies
-
89ace5d7
by Benjamin Schubert
at 2018-11-01T11:16:36Z
Merge branch 'bschubert/remove-pytest-runner' into 'master'
Remove dependency on pytest-runner
See merge request BuildStream/buildstream!886
-
4cfabce8
by Angelos Evripiotis
at 2018-11-01T11:35:02Z
BREAK: manual.yaml: don't set any default env vars
Remove the 'MAKEFLAGS' and 'V' defaults.
Now that there is a specialised 'make' element, these make-specific
defaults don't make sense here. This element is meant to be a blank
slate for folks to build on.
Note that this is a breaking change, that will require some users to
make changes to their .bst files if they are expecting these environment
variable to be set.
_versions.py: BST_FORMAT_VERSION bumped to 18.
This fixes issue #718
-
48860aac
by Tristan Van Berkom
at 2018-11-01T12:01:04Z
Merge branch 'tristan/empty-manual-element' into 'master'
BREAK: manual.yaml: don't set any default env vars
Closes #718
See merge request BuildStream/buildstream!916
-
d868b409
by Daniel Silverstone
at 2018-11-01T13:40:24Z
_yaml.py: Implement `get()` for `ChainMap`
Since the core Python `ChainMap.get()` implements with:
self[key] if key in self else default
The double-chain-lookup is expensive. This simple change solves that
for our ChainMap derived structure. As such it improves matters
for #466 somewhat.
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
7f79b9ce
by Tristan Van Berkom
at 2018-11-01T14:25:57Z
Merge branch 'danielsilverstone-ct/improve-chainmap-get' into 'master'
_yaml.py: Implement `get()` for `ChainMap`
See merge request BuildStream/buildstream!919
-
de59ebdb
by ctolentino8
at 2018-11-02T16:41:54Z
plugins/sources/pip.py: Accomodate characters '-','.','_' for packages
-
8d7cf806
by ctolentino8
at 2018-11-02T16:41:54Z
tests/integration/project/files: Remove pypi-repo
-
9c2f9bf7
by Chandan Singh
at 2018-11-02T17:09:46Z
Merge branch 'chiaratolentino/fix-pip-source-regex' into 'master'
plugins/sources/pip.py: Accomodate characters '-','.','_' for packages
See merge request BuildStream/buildstream!914
-
3788e701
by Jürg Billeter
at 2018-11-03T11:52:00Z
element.py: Pass bare_directory also to SandboxRemote()
Fixes: 825963b5 ("element: Make "--sysroot" take a bare directory")
-
82e971ef
by Jürg Billeter
at 2018-11-05T11:33:20Z
Merge branch 'juerg/remote-execution' into 'master'
element.py: Pass bare_directory also to SandboxRemote()
See merge request BuildStream/buildstream!928
-
62942bfd
by Valentin David
at 2018-11-05T12:14:20Z
Use less fetchers in overnight build to work-around limitations from servers.
Issue happens with sourceware.org server.
Fixes #742
-
442da2f9
by Javier Jardón
at 2018-11-05T12:41:54Z
Merge branch 'valentindavid/limit_fetchers_in_overnight_build' into 'master'
Use less fetchers in overnight build to work-around limitations from servers.
Closes #742
See merge request BuildStream/buildstream!929
-
0993de53
by Richard Maw
at 2018-11-05T16:41:05Z
dev-requirements: Update pytest dependency to 3.8
"_frontend/app.py: Set correct element-path in interactive bst-init"
added a dependency on the tmp_path test fixture, which was added by 3.8.
-
be8f0a54
by richardmaw-codethink
at 2018-11-05T17:08:43Z
Merge branch 'richardmaw/pytest-dep-update' into 'master'
dev-requirements: Update pytest dependency to 3.8
See merge request BuildStream/buildstream!930
-
bfb639bf
by Jürg Billeter
at 2018-11-05T17:18:12Z
element.py: Remove artifacts parameter from constructor
Get the artifact cache from the context instead.
-
ca855f91
by Jürg Billeter
at 2018-11-05T17:18:12Z
_elementfactory.py: Remove unused artifacts parameter from create()
-
15fed21c
by Jürg Billeter
at 2018-11-05T17:18:12Z
_project.py: Remove unused artifacts parameter from create_element()
-
0085d2aa
by Jürg Billeter
at 2018-11-05T17:18:12Z
element.py: Remove unused artifacts parameter from _new_from_meta()
-
f69b1117
by Jürg Billeter
at 2018-11-05T17:18:12Z
_project.py: Remove unused artifacts parameter from load_elements()
-
e398f877
by Jürg Billeter
at 2018-11-05T17:18:12Z
element.py: Do not include type name of artifact cache in cache key
The artifact cache backend does not affect build outputs and we anyway
no longer have pluggable artifact cache backends. This hardcodes
CASCache instead of removing the entry completely to avoid cache key
changes.
-
626d20ae
by Jürg Billeter
at 2018-11-05T17:18:12Z
Split up artifact cache and CAS cache
This changes CASCache from a subclass to a delegate object of
ArtifactCache. As the lower layer, CASCache no longer deals with
elements or projects.
Fixes #659.
-
ec04446b
by Jürg Billeter
at 2018-11-05T17:51:39Z
Merge branch 'juerg/cas' into 'master'
Split up artifact cache and CAS cache
Closes #659
See merge request BuildStream/buildstream!922
-
9a045080
by Josh Smith
at 2018-11-06T13:23:19Z
utils: Fix _pretty_size() for sizes > 1024T
-
d3a07e6b
by Josh Smith
at 2018-11-06T13:23:19Z
Add regression test for _pretty_size
-
0c09fb9c
by richardmaw-codethink
at 2018-11-06T14:12:20Z
Merge branch 'Qinusty/unit-test-utils' into 'master'
Fix issue with _pretty_size with large numbers of bytes
See merge request BuildStream/buildstream!799
-
b4eec489
by Jim MacArthur
at 2018-11-06T16:44:14Z
_sandboxremote.py: Use the standard SandboxError.
Replaces the custom one which was erroneously added during development.
Fixes #746.
-
1f7acf74
by Jim MacArthur
at 2018-11-06T17:16:31Z
Merge branch 'jmac/remote-sandbox-errors' into 'master'
_sandboxremote.py: Use the standard SandboxError
Closes #746
See merge request BuildStream/buildstream!931
-
8e64ccef
by James Ennis
at 2018-11-07T11:06:11Z
using_config.rst: Update our instructions for how we declare caches in user config
-
09ab676d
by James Ennis
at 2018-11-07T11:06:11Z
format_project.rst: Update instructions regarding how we declare remote caches in project.conf
-
f514124f
by James Ennis
at 2018-11-07T11:06:11Z
using_configuring_artifact_server.rst: Remove examples and redirect to user config/project config
-
029ba17d
by richardmaw-codethink
at 2018-11-07T11:32:20Z
Merge branch 'jennis/docs_patch' into 'master'
Update our instructions/examples regarding how we declare remote caches
See merge request BuildStream/buildstream!932
-
261e2cd3
by Jim MacArthur
at 2018-11-07T12:02:43Z
element.py: Use _force_rmtree instead of custom code.
There are arguments for and against using onerror - see issue #735
for details.
-
8931e42c
by Jim MacArthur
at 2018-11-07T12:26:06Z
Merge branch 'jmac/make_writable_combination' into 'master'
Use _force_rmtree instead of custom code.
Closes #735
See merge request BuildStream/buildstream!927
-
6ccfab0b
by Valentin David
at 2018-11-07T16:28:47Z
Fix cache corruption by scripts when layout and integration commands are used
Root directory was marked as a non-artifact mount, so not using
SafeHardLink. However integration commands executed with write access
to the root directory.
Fixes #749
-
f24e20e9
by richardmaw-codethink
at 2018-11-07T16:59:50Z
Merge branch 'valentindavid/script-artifact-corruption' into 'master'
Fix cache corruption by scripts when layout and integration commands are used
Closes #749
See merge request BuildStream/buildstream!934
-
e28f668a
by Valentin David
at 2018-11-08T09:34:11Z
Fix infinite recursion in default strip debug command
Fixes #645.
-
83d15350
by Valentin David
at 2018-11-08T10:01:39Z
Merge branch 'valentindavid/fix-recursive-strip-debug' into 'master'
Fix infinite recursion in default strip debug command
Closes #645
See merge request BuildStream/buildstream!918
-
9f0e12f1
by Benjamin Schubert
at 2018-11-08T10:21:12Z
Move bwrap checks in platform/linux.py
Remove the bwraps checks from _site.py and put them in platform.linux
which is the only place where they are run.
This allows the removal of a double level of caching, making
reasoning about tests easier
-
cf2e0059
by Benjamin Schubert
at 2018-11-08T10:21:12Z
conftest.py: Ensure platform is not maintained between tests
This removes the `_instance` on the platform object that we use
for caching and not recreating the object everytime at the start
of every test.
This is to ensure our tests share the least amount of state.
The performance penalty is from 5 to 10% accross the whole test suite.
The readings were done 5 times for each before and after the change
and on the same computer.
-
c51ba01b
by Benjamin Schubert
at 2018-11-08T10:21:12Z
Test that helpful messages are raised when missing dependencies
This adds a `reason` to the SandboxEror thrown in sandboxdummy
to be able to understand where the error comes from
-
f116b9b7
by Benjamin Schubert
at 2018-11-08T11:07:12Z
Merge branch 'bschubert/tests-no-chroot-linux' into 'master'
test behavior when there is no sandbox on Linux
Closes #696 and #736
See merge request BuildStream/buildstream!921
-
cc2e6ae5
by Valentin David
at 2018-11-08T12:41:36Z
Fix bug with root mounted as non-artifact in script plugin.
The issue was introduced by 6ccfab0b1b25990e406446d5cbe5aee83a5e158a.
-
e578a89f
by Valentin David
at 2018-11-08T13:11:10Z
Merge branch 'valentindavid/fix-script-install-root' into 'master'
Fix bug with root mounted as non-artifact in script plugin.
Closes #756 and #749
See merge request BuildStream/buildstream!941
-
c0a8bb66
by Angelos Evripiotis
at 2018-11-08T15:49:16Z
contributing: WIP again to be kind to reviewers
As someone coming from GitHub to GitLab, I was pleasantly surprised by
the 'filter by WIP status' option.
To make sure we get the most out of it, add a guideline to keep the
filter clean, to reduce reviewer burden.
-
f7231e90
by Angelos Evripiotis
at 2018-11-08T15:49:16Z
contributing: non-WIP should always be landable
I was recently surprised that we don't prefer addressing review comments
in 'fixup!' commits.
Coming from GitHub, I've found that fixup commits make it easier for
reviewers to see what has changed since their last review. The idea is
to use '--autosquash' before landing to clean up the history again.
It's a pleasant surprise that it's easy to keep track of what changed in
merge-requests between pushes, so we can always keep the history clean.
Document this, so folks like me can see the light sooner.
-
f7643440
by Angelos Evripiotis
at 2018-11-08T15:49:16Z
contributing: emphasise the 'why' in commits
Explain why it's useful to mention the decisions made for a change and
provide links for background, e.g. issue numbers.
My general experience of git histories is that folks don't do enough
explaining, so it's great we have a contributing section for it.
I've found it's easier for folks to stick to rules when they know the
practical reasons for them, so provide some.
-
dd5e7b04
by Angelos Evripiotis
at 2018-11-08T16:17:04Z
Merge branch 'aevri/contributing_gitlab' into 'master'
Add more to GitLab-relevant parts of contributing
See merge request BuildStream/buildstream!935
-
fe33e328
by James Ennis
at 2018-11-08T17:54:18Z
using_config.rst: Add documentation to showing how to impose quotas on the local cache
This patch partially resolves #700
-
09faf002
by James Ennis
at 2018-11-08T17:54:18Z
artifactcache.py: Fix misleading error message when using % cache quota
Due to the changed Exception message, this patch also changes
the test_parse_size_over_1024T test in misc.py
-
d153453c
by Javier Jardón
at 2018-11-08T18:22:54Z
Merge branch 'jennis/quota_declaration_fix' into 'master'
Add local cache expiry documentation and fix misleading error message when specifying a percentage cache quota
Closes #700
See merge request BuildStream/buildstream!939
-
7738f6df
by Daniel Silverstone
at 2018-11-09T11:25:44Z
plugin.py: Redirect DEBUG from `__del__` to `sys.stderr`
Previously, debug messages from `Plugin.__del__()` went to stdout
which was a problem for the test suite in extremely rare and hard to
predict circumstances. This corrects that by sending the message to
stderr where all the rest of the messages tend to go.
Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
-
62f59eaa
by Valentin David
at 2018-11-09T13:30:06Z
Merge branch 'danielsilverstone-ct/plugin-destroy-debug-to-stderr' into 'master'
plugin.py: Redirect DEBUG from `__del__` to `sys.stderr`
See merge request BuildStream/buildstream!944
-
35ec6b6a
by Jim MacArthur
at 2018-11-09T14:58:22Z
_casbaseddirectory.py: Enable direct CAS-to-CAS import.
Fixes issue #574.
-
b93100ec
by Jim MacArthur
at 2018-11-09T14:58:22Z
_casbaseddirectory: Fix order of list_modified_paths
This will now return paths in the same order as list_relative_paths.
-
78691fa0
by Jim MacArthur
at 2018-11-09T14:58:22Z
virtual_directory_import.py: New test
-
7ce6581b
by Jim MacArthur
at 2018-11-09T15:25:46Z
Merge branch 'jmac/cas_to_cas_v2' into 'master'
Direct CAS-to-CAS import
Closes #574
See merge request BuildStream/buildstream!911
-
6f4351ec
by Benjamin Schubert
at 2018-11-09T16:53:42Z
source.py: don't let StopIteration propagate to silence() contextmanager
As per PEP 0479 (https://www.python.org/dev/peps/pep-0479/),
StopIteration thrown in context managers are not valid starting from
Python 3.7.
-
e6e03451
by richardmaw-codethink
at 2018-11-12T11:05:15Z
Merge branch 'bschubert/fix-silence-stopiteration-pep-0479' into 'master'
source.py: don't let StopIteration propagate to silence() contextmanager
See merge request BuildStream/buildstream!945
-
ae5ccd76
by Phillip Smyth
at 2018-11-12T14:57:51Z
_project.py: Validate nodes early in Project._load
_project.py: Added validate_nodes() helper function to prevent duplicate lists
element-path was being used before node validation resulting in uncaught errors
-
56a54161
by Phillip Smyth
at 2018-11-12T14:57:51Z
tests/frontend: Add test for invalid element-path
-
e55a9703
by Jürg Billeter
at 2018-11-12T15:44:45Z
Merge branch 'element-path_not_validated' into 'master'
Element path not validated before use
See merge request BuildStream/buildstream!937
-
c87d6e45
by Valentin David
at 2018-11-14T11:43:33Z
Run tests on aarch64
Fixes #755
-
16b01489
by Valentin David
at 2018-11-14T11:43:33Z
Disable cachekey tests on other architectures than x86_64
-
85046b29
by Valentin David
at 2018-11-14T11:43:33Z
Disable tests on example on other architectures than x86_64
-
dd5a073b
by Valentin David
at 2018-11-14T11:43:33Z
Add support for aarch64 in a test
-
c5d72ae7
by Valentin David
at 2018-11-14T11:52:53Z
Test aarch64 only overnight
-
264a57f6
by Javier Jardón
at 2018-11-14T12:22:03Z
Merge branch 'valentindavid/integration-tests-aarch64' into 'master'
Run tests on aarch64
Closes #369 and #755
See merge request BuildStream/buildstream!948
-
da735e56
by Richard Maw
at 2018-11-14T13:30:34Z
_platform/linux.py: Move get_bwrap_version into _site.py
It's inconvenient to have to create a Linux platform to parse the bwrap version
and we want to get the version in a consistent manner.
-
e7633500
by Richard Maw
at 2018-11-14T13:30:34Z
buildstream/sandbox/_sandboxbwrap.py: Distinguish sandbox failure from command failure
If `bwrap` fails to set up the sandbox and start the payload command
it won't write an exit-code in --json-status-fd,
so we can report if it was a sandboxing failure if we don't get exit-code status
and a payload command failure if we do and it's non-zero.
Closes https://gitlab.com/BuildStream/buildstream/issues/286
-
e9e08823
by Richard Maw
at 2018-11-14T13:30:34Z
tests/testutils/site.py: Check for bwrap supporting --json-status-fd
-
90ca007e
by Richard Maw
at 2018-11-14T13:30:34Z
tests/integration/sandbox-bwrap.py: Test distinguishing sandbox exit code from command
-
327b19dd
by richardmaw-codethink
at 2018-11-14T13:59:16Z
Merge branch 'richardmaw/distinguish-sandboxing-build-fail' into 'master'
Distinguish between bubblewrap sandboxing failure and command failure
Closes #286
See merge request BuildStream/buildstream!868
-
082b3811
by Angelos Evripiotis
at 2018-11-15T19:48:12Z
contributing: add guidance on unit tests
Decrease uncertainty around whether unit tests are welcome in the
project or not.
-
b5b79056
by Jürg Billeter
at 2018-11-15T20:24:58Z
Merge branch 'aevri/unit_tests' into 'master'
contributing: add guidance on unit tests
See merge request BuildStream/buildstream!943
-
199bfff1
by Tom Pollard
at 2018-11-17T13:07:59Z
Add cli main and user config option for 'pull-buildtrees' context.
_context.py: Add pull_buildtrees global user context, the default
of which is set to False via the addition of pull-buildtrees to
userconfig.yaml cache group.
_frontend/app.py & cli.py: Add --pull-buildtrees as a bst main
option, which when passed will override the default or user defined
context for pull_buildtrees.
tests/completions/completions.py: Update for the added flag.
-
454dcaf1
by Tom Pollard
at 2018-11-17T13:07:59Z
Don't pull artifact build trees by default.
The addition of cached build trees being included in element
artifacts has led to mostly redundant download overheads when
pulling from a remote artifact server. As such the default behaviour
of pull shouldn't fetch the build tree object if available.
element.py: extend relevant pull logic for specified subdir
consideration and ensure push logic does not lead to partial
artifact pushes. Change assumption that the buildtree can be
extracted if artifact is cached. __cached_buildtree() and
__pull_directories() helpers added.
_artifactcache/: artifactcache.py & cascache.py inclusion of
helper functions for subdir artifact checking & fetching, fetch
logic extended to only pull required artifact directories.
extract & checkout updated to handle full/partial operation.
tests/: addition of integration test pullbuildtrees.py,
buildtrees.py adapted cli options, testutils/artifactshare.py
has_artifact changed to return artifact digest on true condition.
-
0d309478
by Jürg Billeter
at 2018-11-17T13:49:58Z
Merge branch 'tpollard/494' into 'master'
Don't pull artifact buildtrees by default
See merge request BuildStream/buildstream!786
-
27932739
by Benjamin Schubert
at 2018-11-19T09:46:39Z
plugins/sources/pip.py: also look for python version named "python"
In some virtual environments, we might not have the binary with the
exact python version, which would lead the current pip plugin to
fail to find a working pip version.
This adds "python" at the start of the list of valid python versions
-
ea2de561
by Benjamin Schubert
at 2018-11-19T10:19:24Z
Merge branch 'bschubert/fix-pip-python' into 'master'
plugins/sources/pip.py: also look for python version named "python"
Closes #758
See merge request BuildStream/buildstream!942
-
f23b6031
by Benjamin Schubert
at 2018-11-19T11:39:51Z
tests/frontend/buildtrack.py: check for success after invocation
Previously the result of the first call in test_build_track was not
checked to be successful, which would lead to potential erros later on
that were hard to debug.
Let's check it everytime
-
a6defc0b
by Benjamin Schubert
at 2018-11-19T11:39:51Z
Fix os.rename in git source element to correctly handle error codes
According to the documentation
(https://www.unix.com/man-page/POSIX/3posix/rename/), when the directory
already is there, either EEXIST or ENOTEMPTY could be thrown.
Previously only ENOTEMPTY was checked.
Done:
- Separated the move into its own function
- Check for both errors
- Create unit tests for it, covering most test cases
-
88089d2d
by Benjamin Schubert
at 2018-11-19T11:39:51Z
Extract atomic move function to utils.py
Moving atomically a file/directory can be tricky since different
errors might be raised for the same underlying problem.
Having a utility function to reduce this discrepancies will help
in ensuring we have correct behavior
-
fd9e46be
by Benjamin Schubert
at 2018-11-19T11:39:51Z
cascache.py: use move_atomic instead of manual error checking
-
d32e0b83
by Benjamin Schubert
at 2018-11-19T11:39:51Z
sources/pip.py: Use move_atomic instead of manual os.rename
This uses move_atomic insteand of the manual os.rename and manual error
checking and throws a SourceError for consistency with other modules.
-
6f837118
by Benjamin Schubert
at 2018-11-19T12:22:40Z
Merge branch 'bschubert/fix-atomic-move-git-repo' into 'master'
Fix os.rename in git source element to correctly handle error codes
See merge request BuildStream/buildstream!938
-
30b72244
by Chandan Singh
at 2018-11-19T12:35:01Z
buildstream/_pipeline.py: Simplify error message for assert_consistent()
Currently, `Pipeline.assert_consistent()` prints an error message like
the one below when certain elements have inconsistent sources:
```
...
Element: hello.bst is inconsistent
Source tar source at hello.bst [line 16 column 2] is missing ref
...
```
Drop the word "Source" from the beginning of the message as
"Source <source-kind> source ..." is kind of awkward to read.
-
76c5d2f8
by Chandan Singh
at 2018-11-19T12:35:01Z
Add `bst source-checkout` command
As discussed in
https://mail.gnome.org/archives/buildstream-list/2018-September/msg00064.html,
add `bst source-checkout` command. This will allow users to checkout
sources for a given target.
* _frontend/cli.py: Add source-checkout command
* _pipeline.py: Add assert_sources_cached() method
* _stream.py: Add source_checkout method, abstract out
__check_location_writable() method that used to part of checkout()
-
9f629638
by Chandan Singh
at 2018-11-19T13:03:36Z
Merge branch 'chandan/source-checkout' into 'master'
Add `bst source-checkout` command
See merge request BuildStream/buildstream!820
-
bc827cc8
by Abderrahim Kitouni
at 2018-11-19T14:55:15Z
plugins/elements/cmake.yaml: always specify variable types
cmake sometimes misinterprets relative paths as relative to the current directory
if this is not specified. See freedesktop-sdk/freedesktop-sdk#431
adjust tests/format/variables.py accordingly.
-
5fbc5f41
by Valentin David
at 2018-11-19T15:31:05Z
Merge branch 'abderrahim/cmake-variable-types' into 'master'
plugins/elements/cmake.yaml: always specify variable types
Closes #761
See merge request BuildStream/buildstream!947
-
06e28860
by Benjamin Schubert
at 2018-11-19T15:52:24Z
Don't cache sandbox errors
Sandbox errors (like missing host tools) are dependent on the host
system and rarely on what is actually done.
It is therefore better to not cache them as they are subject to
change between two runs.
Also add test to ensure sandbox failure are not cached
-
dd36cfbc
by Benjamin Schubert
at 2018-11-19T15:52:24Z
element.py: Only cache on element error or success
Reverse the way we were handling caching by only caching
when we know it's a problem with the build or the build is a
success in order not to cache transient errors.
-
eac7274d
by Benjamin Schubert
at 2018-11-19T16:29:07Z
Merge branch 'bschubert/dont-cache-errors-from-host-tools' into 'master'
Don't cache sandbox failures
Closes #727
See merge request BuildStream/buildstream!895
-
625dfe1f
by Chandan Singh
at 2018-11-19T16:41:17Z
NEWS: Add entry for the new source-checkout command
`bst source-checkout` command was recently added in
https://gitlab.com/BuildStream/buildstream/merge_requests/820. Add a
NEWS entry for the new command.
-
d4f12184
by Chandan Singh
at 2018-11-19T17:15:10Z
Merge branch 'chandan/source-checkout-news' into 'master'
NEWS: Add entry for the new source-checkout command
See merge request BuildStream/buildstream!960
-
ba6c96e8
by Richard Maw
at 2018-11-19T19:54:30Z
CasBasedDirectory: Change constructor to take a CASCache instead of a Context
The Context was only used to obtain a reference to the CASCache
and set the unused cas_directory field.
-
8a0dc3a3
by Jürg Billeter
at 2018-11-19T19:54:30Z
tests/testutils/artifactshare.py: Do not create a fake context
The fake context did not set the cache quota, triggering an error. With
CASCache now separate from ArtifactCache, we can instantiate a CASCache
without context.
-
8722aced
by Jürg Billeter
at 2018-11-19T19:54:30Z
tests/plugins/pipeline.py: Load context default values
-
fc56ffa4
by Jürg Billeter
at 2018-11-19T19:54:30Z
_context.py: Drop duplicated default values for user configuration
The default values are in userconfig.yaml, together with the
documentation. The default values should not be duplicated in
_context.py.
-
3f663d82
by Jürg Billeter
at 2018-11-19T20:36:57Z
Merge branch 'juerg/context-default-values' into 'master'
_context.py: Drop duplicated default values for user configuration
See merge request BuildStream/buildstream!953
-
b498cce7
by Angelos Evripiotis
at 2018-11-20T11:17:38Z
_yaml: document node_get()'s default_value arg
-
eb2d376f
by Angelos Evripiotis
at 2018-11-20T11:17:38Z
_context: allow 'terminate' for scheduler.on-error
Enable this option of 'terminate', which is mentioned in userconfig.yaml
and handled in _frontend/app.py:_handle_failure(). It appears to have
been left out of the valid_actions as an oversight.
Originally introduced in
https://gitlab.com/BuildStream/buildstream/commit/2622d5da9bd9fefd87436613d6e3e9770fdd0f28
-
b81c4333
by Angelos Evripiotis
at 2018-11-20T11:17:38Z
_context: refactor, extract _node_get_option_str
Use a new helper function to simplify working with nodes that can only
accept certain strings. This will be used when adding the prompt.*
config options.
In later work we can see if this function would be useful elsewhere, and
could be added to '_yaml.py'.
-
27ca6593
by Angelos Evripiotis
at 2018-11-20T11:19:33Z
Add prompt.auto-init buildstream.conf option
Provide an option in buildstream.conf to disable the 'Would you like to
...' prompt when we cannot resolve a project.
Some users prefer not to be interrupted by such prompts, so pave the way
to creating options to disable all those that might get in the way.
Follow the example of the advice.* options 'git-config', and create a
namespace for these UI options grouped by behaviour, rather than an
over-reaching 'ui.*' namespace. In later work perhaps we'll also add
'advice.*' options.
Add a NEWS item for this.
-
7ae3a3d2
by Angelos Evripiotis
at 2018-11-20T11:43:49Z
Add prompt.workspace-... options
Provide options in project.conf to disable the 'Are you sure ...'
prompts when making destructive changes:
- Add prompt.really-workspace-close-remove-dir
- Add prompt.really-workspace-reset-hard
Add a NEWS item for these.
-
8071c00c
by Angelos Evripiotis
at 2018-11-20T12:12:11Z
Merge branch 'aevri/prompt-config' into 'master'
Add 'prompt.*' config options to buildstream.conf
See merge request BuildStream/buildstream!887
-
fd41b0b5
by Raoul Hidalgo Charman
at 2018-11-20T16:32:10Z
_sandboxremote.py: Add sigterm handler that sends CancelOperation
-
c6306b88
by Jürg Billeter
at 2018-11-20T16:59:45Z
Merge branch '725-job-cancellation-on-remote-builds' into 'master'
_sandboxremote.py: Add sigterm handler that sends CancelOperation
Closes #725
See merge request BuildStream/buildstream!900
-
9fa8a881
by James Ennis
at 2018-11-20T17:00:42Z
_yaml.py: Fix incorrect error message
This patch ensures that we receive an appropriate error
message if we specify an absolute path that leads within
the project.
-
fb8860db
by Jürg Billeter
at 2018-11-20T17:32:58Z
Merge branch 'jennis/less_restrictive_pathways' into 'master'
Fix warning when using an absolute path which exists inside the project
See merge request BuildStream/buildstream!956
-
87751350
by Benjamin Schubert
at 2018-11-20T18:10:48Z
gitlab-ci.yml: Use CI_PROJECT_DIR instead of $(pwd) for cache
-
8668df9f
by Benjamin Schubert
at 2018-11-20T18:10:48Z
Export variables globally for less repetition.
This doesn't change the test behavior and enable for better
composability of tests
-
cc6a7f0d
by Benjamin Schubert
at 2018-11-20T18:10:48Z
gitlab-cy.yml: make linux-tests-template more composable.
This will allow for further de-duplication of work.
-
eb37dd89
by Benjamin Schubert
at 2018-11-20T18:10:48Z
gitlab-ci.yml: remove unused artifacts for unix tests
The unix-logs is always empty, there is therefore no reasons of keeping
it there
-
e030b48e
by Benjamin Schubert
at 2018-11-20T18:10:48Z
gitlab-ci.yml: use template for unix tests
This makes all tests use the same template, which makes reasoning
about them simpler
-
57e6358f
by Benjamin Schubert
at 2018-11-20T18:10:48Z
Ensure tests requiring bubblewrap are correctly marked
-
e70dd1f8
by Benjamin Schubert
at 2018-11-20T18:10:48Z
gitlab-ci.yml: Add linux without brwap/ostree to ensure tests behave
This allows us to make sure that tests behave nicely while missing brwap
and ostree.
-
f7cb4386
by Benjamin Schubert
at 2018-11-20T18:10:48Z
gitlab-ci.yml: Extract test command as a variable
This removes the need of having it synchronized in multiple places
-
7f316a76
by Benjamin Schubert
at 2018-11-20T18:41:03Z
Merge branch 'bschubert/add-fedora-nobwrap-tests' into 'master'
Add tests for fedora with no bubblewrap available
See merge request BuildStream/buildstream!949
-
a50c598e
by Benjamin Schubert
at 2018-11-21T09:24:15Z
utils.py: Ensure move_atomic booleans are keyword only arguments
This is to be coherent with the rest of the codebase
-
b2f604b8
by Benjamin Schubert
at 2018-11-21T09:24:15Z
utils.py: improve documentation of move_atomic
-
daca0c01
by Benjamin Schubert
at 2018-11-21T09:52:42Z
Merge branch 'bschubert/mr938-comments' into 'master'
Followup on MR 938, addressing additional comments
See merge request BuildStream/buildstream!958
-
050cd1bd
by Tom Pollard
at 2018-11-21T11:37:48Z
tests/plugin/pipeline.py: Avoid using host user conf
create_pipeline was passing no config parameter to context.load,
leading to _context.py loading in the host's buildstream user conf
when running tests locally, potentially leading to unwanted errors.
-
58353aad
by Jürg Billeter
at 2018-11-21T14:29:49Z
Merge branch 'tpollard/pipelinehostconfig' into 'master'
tests/plugin/pipeline.py: Avoid using host user conf
See merge request BuildStream/buildstream!961
-
2022c3e8
by William Salmon
at 2018-11-21T15:53:03Z
tests/frontend/workspace.py: Refactor support function
The Refactor now makes it possible to test for the forth coming multi
element workspace open commands and to test for failure.
-
7bdd44b2
by William Salmon
at 2018-11-21T15:53:03Z
Workspace CLI update
This is to update the workspace CLI to as agreed on the mailing list
https://mail.gnome.org/archives/buildstream-list/2018-September/msg00046.html
This patch also introduces the default workspace directory.
-
6612e708
by William Salmon
at 2018-11-21T15:53:03Z
Added a news item for workspace open changes
-
4d6e2cd3
by Will Salmon
at 2018-11-21T16:20:26Z
Merge branch 'willsalmon/defaultWorkspaces' into 'master'
Updated Workspace CLI
See merge request BuildStream/buildstream!897
-
dddf0eaa
by Valentin David
at 2018-11-22T11:12:53Z
.gitlab-ci.yml: Bump version of fd.o SDK and bst_external in overnight tests
-
abef70fe
by Javier Jardón
at 2018-11-22T11:41:31Z
Merge branch 'valentindavid/overnight-tests-bump-fdo-sdk-version' into 'master'
Bump version of fd.o SDK and bst_external in overnight tests
See merge request BuildStream/buildstream!962
-
1e3a5bb1
by Javier Jardón
at 2018-11-26T01:54:15Z
Use pycodestyle instead pep8
pep8 package was renamed to pycodestyle to reduce confusion.
-
afaff171
by Javier Jardón
at 2018-11-26T01:54:15Z
setup.cfg: Ignore W605 and W504 for now
This is the current occurence:
40 W605 invalid escape sequence '\s'
75 W504 line break after binary operator
-
f4a3a23f
by Javier Jardón
at 2018-11-26T01:56:43Z
WIP
-
4c471267
by Javier Jardón
at 2018-11-26T02:02:40Z
WIP