-
ac995236
by Tom Pollard
at 2018-12-20T17:18:02Z
_stream.py: fix _buildtree_pull_required logic
-
c3153dea
by Tom Pollard
at 2018-12-21T10:10:58Z
Merge branch 'tpollard/fixbuildtreereq' into 'master'
_stream.py: fix _buildtree_pull_required logic
See merge request BuildStream/buildstream!1021
-
caf49669
by Tristan Van Berkom
at 2018-12-26T19:54:05Z
dev-requirements.txt: Require at least pytest 3.9
Recently we have been adding some tests which use the new tmp_path
fixture from pytest, this is new in 3.9.
-
425781db
by Tristan Van Berkom
at 2018-12-26T19:54:05Z
_signals.py: Fixing new (simplifiable-if-_expression_) linter errors
-
1f8cc996
by Tristan Van Berkom
at 2018-12-26T19:54:05Z
element.py: Fixing new (unnecessary-pass) and (simplifiable-if-_expression_) linter errors
-
89632150
by Tristan Van Berkom
at 2018-12-26T19:54:05Z
source.py: Fixing new (unnecessary-pass) linter error
-
26d135ba
by Tristan Van Berkom
at 2018-12-26T19:54:05Z
_artifactcache/casserver.py: Fixed new (wrong-import-order) linter error
-
d0988f80
by Tristan Van Berkom
at 2018-12-26T19:54:05Z
_frontend/cli.py: Fixing new (unnecessary-pass) linter errors
-
4d54ab78
by Tristan Van Berkom
at 2018-12-26T19:54:05Z
_frontend/complete.py: Fixing new (simplifiable-if-_expression_) linter error
-
ee0d5bf6
by Tristan Van Berkom
at 2018-12-26T19:54:05Z
_frontend/widget.py: Fixing new (wrong-import-order) linter error
-
36295282
by Tristan Van Berkom
at 2018-12-26T19:54:05Z
plugins/elements/import.py: Fixing new (duplicate-string-formatting-argument) linter error
-
4ffe03c6
by Tristan Van Berkom
at 2018-12-26T19:54:05Z
buildstream/storage/_casbaseddirectory.py: Fixing new (unnecessary-pass) linter errors
-
80c137d5
by Tristan Van Berkom
at 2018-12-26T19:54:05Z
dev-requirements.txt: Require new version of pytest-datafiles
Since Thomas Nixon fixed the datafiles plugin to be compatible
with pytest >= 3.8.0, we can now depend on it and avoid the
warnings and errors which the older datafiles plugin causes.
This fixes issue #636
-
61906a14
by Tristan Van Berkom
at 2018-12-26T19:54:05Z
dev-requirements.txt: Remove the restriction on pylint.
After having fixed the more recent linting errors, the
tests should pass with recent versions of pylint. Tested
with pylint 2.2.2.
-
94b9948c
by Tristan Van Berkom
at 2018-12-26T22:35:29Z
Merge branch 'tristan/test-fixes' into 'master'
Misc fixes to the tests and newer linter errors
Closes #636
See merge request BuildStream/buildstream!1011
-
fe4fc0b5
by Tristan Van Berkom
at 2018-12-26T23:19:07Z
_downloadablefilesource.py: Avoid crashes when HOME is unset
The python netrc module will raise OSError in the case
that HOME is not set, this was discovered while running
tests under tox.
-
c61e1793
by Tristan Van Berkom
at 2018-12-26T23:19:07Z
tests/testutils/runcli.py: Fixed broken environment handling
Treat None values in the passed dictionary as keys to be removed
from the environment, this was already happening at restoration time.
-
50c1bb7f
by Tristan Van Berkom
at 2018-12-26T23:19:07Z
tests/sources/tar.py: Test that we don't crash when HOME is unset
This happens when the netrc module is searching for a ~/.netrc
file and it doesnt find any HOME set.
-
32a101f6
by Tristan Van Berkom
at 2018-12-26T23:53:29Z
Merge branch 'tristan/fix-netrc-crasher' into 'master'
Fix netrc crasher
See merge request BuildStream/buildstream!1025
-
a080b6c7
by Tristan Van Berkom
at 2018-12-31T19:03:25Z
tests/loader/junctions.py: Use Result checking APIs instead of manually
Better to use the same API everywhere for thecking CLI results.
-
d7c8b54d
by Tristan Van Berkom
at 2018-12-31T19:14:16Z
tests/loader/junctions.py: Test the error we expect when a cross junction element is not found
-
560a6342
by Tristan Van Berkom
at 2018-12-31T19:49:24Z
Merge branch 'tristan/junction-tests' into 'master'
Junction test addition
See merge request BuildStream/buildstream!1029
-
8a020a5a
by Chandan Singh
at 2018-12-31T19:54:25Z
_loader/loader.py: Refactor warnings about element names in one method
Currently we some duplication in the way we check for invalid filenames.
To make it more robust and allow room for adding more warnings, refactor
it into a separate method `_warn_invalid_elements()` that handles just
this.
-
87568074
by Chandan Singh
at 2018-12-31T19:54:25Z
_loader/loader.py: Add warnings about invalid characters in filename
See https://mail.gnome.org/archives/buildstream-list/2018-December/msg00061.html
for some related discussion.
-
4b2992c1
by Chandan Singh
at 2018-12-31T19:54:25Z
tests/buildcheckout.py: Add tests for invalid chars in filename
-
ce9e9c2d
by Chandan Singh
at 2018-12-31T19:54:25Z
doc: Add note about valid element names
-
b848172c
by Tristan Van Berkom
at 2018-12-31T20:29:51Z
Merge branch 'chandan/element-filename-requirements' into 'master'
Add warnings about invalid characters in filename
See merge request BuildStream/buildstream!1028
-
bb9df2a1
by Chandan Singh
at 2019-01-02T11:59:35Z
doc/Makefile: Update comment about sphinx entrypoint
https://github.com/sphinx-doc/sphinx/issues/4375 has been closed as
`wontfix`. Update the comment in doc/Makefile that mentions it, based on
the discussion on the issue.
-
43eaf6ee
by Chandan Singh
at 2019-01-02T13:42:10Z
Merge branch 'chandan/update-doc-makefile-note' into 'master'
doc/Makefile: Update comment about sphinx entrypoint
See merge request BuildStream/buildstream!1026
-
3b5c8a28
by Tristan Maat
at 2019-01-02T15:57:13Z
.gitlab-ci.yml: Use new testsuite images with pycodestyle 1.4.0
We need to bump the testsuite image versions to use the new
pycodestyle update that allows excluding file paths, so that we can
exclude generated python code.
-
bd60e8a9
by Javier Jardón
at 2019-01-02T16:59:41Z
Use pycodestyle instead of pep8
pep8 package was renamed to pycodestyle to reduce confusion.
Disabled warnings:
W504 - Line break after binary operator (seems people like this)
W605 - Invalid escape sequence (some of our regexes use these)
-
32c47d1c
by Tristan Maat
at 2019-01-02T17:32:09Z
Merge branch 'jjardon/pycodestyle' into 'master'
Use pycodestyle instead pep8 python module
See merge request BuildStream/buildstream!638
-
1cbd9a73
by Chandan Singh
at 2019-01-02T19:39:58Z
Add requirements files for install, test and plugin dependencies
Add `.in` and `.txt` requirements files for BuildStream's pure python
dependencies. For each pair, the `.in` file is supposed to capture the
loose version requirements, and the corresponding `.txt` file is
supposed to have frozen requirements. We have 3 such sets:
* `requirements`: BuildStream's runtime dependencies
* `dev-requirements`: Dependencies for running tests
* `plugin-requirements`: Dependencies for core plugins
Note that the frozen requirements files will only be used for testing
purposes, and `setup.py` will continue to read loose requirements.
-
6b0cb5f3
by Chandan Singh
at 2019-01-02T19:41:21Z
Add tox.ini to enable running tests using tox
Add `tox.ini` file that will enable us to use
[tox](https://tox.readthedocs.io/) as a frontend for running tests.
Since we share the config via `setup.cfg` and requirements via
requirements files, commands like `python3 setup.py test` will continue
to work.
-
9d2d1d4f
by Chandan Singh
at 2019-01-03T03:31:52Z
.gitlab-ci.yml: Run tests using tox
Instead of invoking tests throung `setup.py`, use `tox` as a frontend in
the CI pipelines.
-
8ae04283
by Tristan Van Berkom
at 2019-01-03T03:31:52Z
CONTRIBUTING.rst: Updated to reflect running tests using tox.
-
e7e2a5c1
by Chandan Singh
at 2019-01-03T03:31:52Z
CONTRIBUTING.rst: Add steps for installing non-python build dependencies
Since we don't allow use of site packages by default in our `tox`
configuration, people will need to install non-python build dependencies
of some of our dependencies that do not provide pre-built wheels. We
have two such packages at the moment:
* psuitl: requires python C headers, compiler
* pygobject: requires python C headers, pkg-config, compiler, cairo
headers, and gobject libraries
-
afa0a369
by Chandan Singh
at 2019-01-03T03:31:52Z
Move all requirements files into "tools" directory
These new `.in` and `.txt` are making the repository look very
cluttered. Move them to a separate `tools` directory to make it look a
bit cleaner.
-
3fae3446
by Chandan Singh
at 2019-01-03T03:44:17Z
Move sphinx build functionality to tox
Currently the CI and the docs both have to duplicate the same inforation
about how to gather dependencies etc, and have to use hacky ways to run
them.
Add a new `docs` environment to our tox setup so that building docs is
as simple as running `tox -e docs`.
-
1f7bc655
by Chandan Singh
at 2019-01-03T03:44:17Z
CONTRIBUTING.rst: Add instructions to recreate tox environment
When we update our dependencies, developers will eventually need to
recreate their `tox` environments to get correct results. This happens
because `tox` isn't particularly good at recognizing changes in the
requirements files.
-
669b55b0
by Tristan Van Berkom
at 2019-01-03T15:47:27Z
Merge branch 'chandan/toxify' into 'master'
Use tox as a frontend for running tests
See merge request BuildStream/buildstream!1027
-
ca547f19
by Chandan Singh
at 2019-01-03T19:34:49Z
tests/frontend/buildcheckout.py: Fix bad filename issue for Windows
In BuildStream/buildstream!1028, we added a test specifically to test
that BuildStream correctly raises an warning when the name of an element
contains characters that are invalid on Windows. Unfortunately, we
didn't see it coming that it would make it impossible to checkout this
branch on Windows.
Fix it by generating this file, only if we are not running on Windows.
* tests/testutils/site.py: Add `IS_WINDOWS` check
* tests/frontend/buildcheckout.py: Generate file with invalid filename
on the fly
* Remove tests/frontend/project/elements/invalid-chars.
Fixes #842.
Note that this may still cause issues on WSL when running tests on a
shared filesystem, but that seems to be a generic issue on WSL with
`os.rename`.
-
01c3761c
by Chandan Singh
at 2019-01-03T20:03:32Z
Merge branch 'chandan/fix-bad-filename-mini-disaster' into 'master'
tests/frontend/buildcheckout.py: Fix bad filename issue for Windows
Closes #842
See merge request BuildStream/buildstream!1032
-
7850e4e2
by Tristan Van Berkom
at 2019-01-03T20:55:00Z
setup.cfg: Specify the pycodestyle configurations separately from pytest configuration
-
3c9d592d
by Tristan Van Berkom
at 2019-01-03T20:55:00Z
setup.cfg: Don't lint by default through setup.py test invocations
-
feba48e7
by Tristan Van Berkom
at 2019-01-03T20:55:00Z
tox.ini: Added new 'lint' environment to lint separately
-
9caa0ea0
by Tristan Van Berkom
at 2019-01-03T20:55:00Z
.gitlab-ci.yml: Update the CI to run the linter separately through tox
This should save us some cycles in CI, and also allow the
developer to more conveniently lint separately from testing.
-
91c8b634
by Tristan Van Berkom
at 2019-01-03T21:04:49Z
CONTRIBUTING.rst: Adding instructions to run the linter separately
-
892bf99e
by Tristan Van Berkom
at 2019-01-03T21:31:04Z
tools/requirements.{in,txt}: Fix warning about duplicate jinja
Apparently you need to spell Jinja2 with a capital J, otherwise
we find it listed twice in the resulting requirements.txt files.
-
7c681c42
by Tristan Van Berkom
at 2019-01-03T21:32:40Z
tools/dev-requirements.{in,txt}: Removed requirements on pylint/codestyle pytest plugins
We no longer run the linters through pytest, so we don't need these
plugins anymore.
-
d6689ed2
by Tristan Van Berkom
at 2019-01-03T22:09:02Z
Merge branch 'tristan/toxic-linting' into 'master'
Run the linter separately from the tests
See merge request BuildStream/buildstream!1033
-
e15278d7
by Thomas Coldrick
at 2019-01-03T22:14:01Z
Use collections.abc for Mapping, Iterable
In _yaml.py and _frontend/complete.py we were getting pylint warnings
for using collections.Mapping and collections.Iterable, which are
abstract classes now provided from collections.abc. This patch just
uses the classes from the right place.
-
a85da591
by Tristan Van Berkom
at 2019-01-03T22:36:16Z
Merge branch 'coldtom/collections' into 'master'
Use collections.abc for Mapping, Iterable
Closes #831
See merge request BuildStream/buildstream!1020
-
cf0e7d26
by Jürg Billeter
at 2019-01-04T07:07:59Z
_frontend/cli.py: Reinstate support for guessing targets
Moving fetch and track to the source command group accidentally dropped
the support for guessing targets for these commands when invoked from a
workspace directory. This brings it back.
-
0751fc8f
by Jürg Billeter
at 2019-01-04T11:27:58Z
Merge branch 'juerg/source-guess-target' into 'master'
_frontend/cli.py: Reinstate support for guessing targets
See merge request BuildStream/buildstream!1036
-
ebd965fe
by Chandan Singh
at 2019-01-04T15:21:35Z
Rename "tools" directory to "requirements"
This seems like a better name for the directory, as it more closely
describes the purpose of its contents.
-
bb712886
by Chandan Singh
at 2019-01-04T15:21:35Z
Add Makefile to update requirements files
-
b9792837
by Chandan Singh
at 2019-01-04T15:21:35Z
CONTRIBUTING.rst: Add instructions to update requirements files
Split the "The MANIFEST.in and setup.py" section in two:
"Managing data files" and "Updating BuildStream's Python dependencies".
Briefly explain the layout of `requirements` directory and add
instructions to use the Makefile added in the last commit.
-
45692335
by Tristan Van Berkom
at 2019-01-04T15:50:14Z
Merge branch 'chandan/update-requirements-one-liner' into 'master'
Add Makefile to update requirements files
See merge request BuildStream/buildstream!1035
-
f42dcb54
by Chandan Singh
at 2019-01-04T17:16:43Z
.gitlab-ci.yml: Remove prepare stage
As we now run tests using `tox`, we don't need to worry about manually
packing and unpacking BuildStream. So, we can remove the preapre stage
entirely.
Update `coverage` and nightly jobs to appropriately cope with this
change. Both these jobs now install all runtime dependencies from
requirements files.
-
ee29f207
by Chandan Singh
at 2019-01-04T17:41:02Z
Merge branch 'chandan/no-sdist' into 'master'
.gitlab-ci.yml: Remove prepare stage
See merge request BuildStream/buildstream!1037
-
f122cfe8
by Jürg Billeter
at 2019-01-06T17:57:33Z
_frontend/app.py: Use buildtree for interactive shell on build failure
Fixes: e29aea36 ("Basic options for shell --build to use buildtrees")
-
7dfb85b3
by Tristan Van Berkom
at 2019-01-06T18:31:25Z
Merge branch 'juerg/shell-buildtree' into 'master'
_frontend/app.py: Use buildtree for interactive shell on build failure
See merge request BuildStream/buildstream!1039
-
891fcb0e
by Tristan Van Berkom
at 2019-01-07T16:47:01Z
Fix stack traces discovered with ^C forceful termination.
* utils.py:_kill_process_tree(): Ignore NoSuchProcess errors
These are caused because we issue SIGTERM, and if the process
has not exited after a timeout, we kill it.
* _scheduler/jobs/job.py: Stop handling NoSuchProcess errors here
redundantly, they are already ignored.
It seems that we were ignoring it after sleeping when terminating
tasks from the scheduler... but we were not ignoring it when performing
the same pattern in the `Plugin.call()` -> `utils._call()` path, so
we were still getting these exceptions at termination time from host
tool processes launched by source plugins.
-
5de42d43
by Tristan Van Berkom
at 2019-01-07T18:00:37Z
Merge branch 'tristan/keyboard-interrupt-stack-trace' into 'master'
Fix stack traces discovered with ^C forceful termination.
See merge request BuildStream/buildstream!1043
-
059035b9
by Tristan Van Berkom
at 2019-01-07T18:02:00Z
_scheduler/scheduler.py: Make _schedule_jobs() private
This is not used anywhere outside of the Scheduler, currently
only the Scheduler itself is allowed to queue a job at this level.
If the highlevel business logic for automatic queueing of auxiliary
jobs moves to another location, we can make this public again.
-
b83d1b1f
by Tristan Van Berkom
at 2019-01-07T18:02:00Z
_scheduler/scheduler.py: Only run one cache size job at a time
When queuing the special cache management related cleanup and
cache size jobs, we now treat these jobs as special and do the
following:
* Avoid queueing a cleanup/cache_size job if one is already queued
We just drop redundantly queued jobs here.
* Ensure that jobs of this type only run one at a time
This could have been done with the Resources mechanics,
however as these special jobs have the same properties and
are basically owned by the Scheduler, it seemed more straight
forward to handle the behaviors of these special jobs together.
This fixes issue #753
-
16a8816f
by Tristan Van Berkom
at 2019-01-07T18:02:00Z
Scheduler: Introduced JobStatus instead of simple success boolean
This changes the deepest callback from when a Job completes to
propagate a JobStatus value instead of a simple boolean, and updates
all of the effected code paths which used to receive a boolean
to now handle the JobStatus values.
This further improves the situation for issue #753, as now we avoid
queueing cache size jobs for pull jobs which are skipped.
-
c2fc2a5e
by Tristan Van Berkom
at 2019-01-07T18:02:00Z
_scheduler/jobs/job.py: Removed 'skipped' property
This is redundant now that we report it through the JobStatus.
-
3e3984ad
by Tristan Van Berkom
at 2019-01-07T18:50:23Z
Merge branch 'tristan/one-cache-size-job' into 'master'
Only queue one cache size job
Closes #753
See merge request BuildStream/buildstream!1040
-
512c726e
by Tristan Van Berkom
at 2019-01-08T03:38:11Z
sandbox/sandbox.py: Fix regression of command logging
Since we added batch commands, the batch commands print the
text of the commands directly in the message text, but this is wrong.
The detail string is the appropriate place for text of unknown lengths
(the user can actually configure how many max lines of commands they
want to see in their log), the message text itself should be controlled
and brief enough to avoid text wrapping.
-
01171988
by Tristan Van Berkom
at 2019-01-08T04:20:14Z
Merge branch 'tristan/fix-command-status-messages' into 'master'
sandbox/sandbox.py: Fix regression of command logging
See merge request BuildStream/buildstream!1044
-
6c1d06d6
by Phil Dawson
at 2019-01-08T10:24:32Z
element.py: remove reference to source bundle command
This command has been replacved by the bst source checkout command
-
914ecb72
by Jürg Billeter
at 2019-01-08T10:54:02Z
Merge branch 'phil/remove-source-bundle-reference' into 'master'
element.py: remove documentation reference to source bundle command
See merge request BuildStream/buildstream!1041
-
81f7a964
by Tristan Van Berkom
at 2019-01-08T16:55:58Z
sandbox/sandbox.py: Use assertions for programming errors instead of BstErrors.
When code is faulty, in a plugin or in the core, we should
get a stack trace and a BUG message.
-
edcc43ed
by Tristan Van Berkom
at 2019-01-08T19:19:10Z
Merge branch 'tristan/sandbox-programming-error' into 'master'
sandbox/sandbox.py: Use assertions for programming errors instead of BstErrors.
See merge request BuildStream/buildstream!1046
-
17c5ca2d
by Valentin David
at 2019-01-09T10:45:23Z
Use relative path to project directory for remote execution certificates/keys
-
605f8d11
by Valentin David
at 2019-01-09T10:45:23Z
Add support for user remote execution configuration
Fixes #631.
-
65ea03ab
by Valentin David
at 2019-01-09T10:45:23Z
Add support for https channel to remote execution and actions servers
Fixes #780.
-
ba05c4fa
by Valentin David
at 2019-01-09T12:58:59Z
Merge branch 'valentindavid/remote_execution_configuration' into 'master'
Remote execution configuration: HTTPS and user configuration
Closes #780 and #631
See merge request BuildStream/buildstream!1030
-
eb0dbcfc
by Valentin David
at 2019-01-09T13:56:58Z
buildstream/plugins/elements/script.py: Mark script as BST_VIRTUAL_DIRECTORY
ScriptElement does not use Sandbox.get_directory. It works using it
with remote execution.
Fixes #850
-
f7681925
by Valentin David
at 2019-01-09T14:44:22Z
Merge branch 'valentindavid/script_virtual_directory' into 'master'
buildstream/plugins/elements/script.py: Mark script as BST_VIRTUAL_DIRECTORY
Closes #850
See merge request BuildStream/buildstream!1047
-
f87d1c93
by Jim MacArthur
at 2019-01-09T15:01:18Z
sandboxremote.py: Rename 'instance_name' option to 'instance-name'
It was 'instance-name' in the documentation.
-
ed8bc5bc
by Jim MacArthur
at 2019-01-09T15:30:28Z
Merge branch 'jmac/rename_instance_name' into 'master'
Rename 'instance_name' option to 'instance-name'
See merge request BuildStream/buildstream!1048
-
ab8312e6
by Benjamin Schubert
at 2019-01-09T20:22:18Z
Add a Centos runner
Centos is apparently different enough from fedora when running tests.
