-
f122cfe8
by Jürg Billeter
at 2019-01-06T17:57:33Z
_frontend/app.py: Use buildtree for interactive shell on build failure
Fixes: e29aea36 ("Basic options for shell --build to use buildtrees")
-
7dfb85b3
by Tristan Van Berkom
at 2019-01-06T18:31:25Z
Merge branch 'juerg/shell-buildtree' into 'master'
_frontend/app.py: Use buildtree for interactive shell on build failure
See merge request BuildStream/buildstream!1039
-
891fcb0e
by Tristan Van Berkom
at 2019-01-07T16:47:01Z
Fix stack traces discovered with ^C forceful termination.
* utils.py:_kill_process_tree(): Ignore NoSuchProcess errors
These are caused because we issue SIGTERM, and if the process
has not exited after a timeout, we kill it.
* _scheduler/jobs/job.py: Stop handling NoSuchProcess errors here
redundantly, they are already ignored.
It seems that we were ignoring it after sleeping when terminating
tasks from the scheduler... but we were not ignoring it when performing
the same pattern in the `Plugin.call()` -> `utils._call()` path, so
we were still getting these exceptions at termination time from host
tool processes launched by source plugins.
-
5de42d43
by Tristan Van Berkom
at 2019-01-07T18:00:37Z
Merge branch 'tristan/keyboard-interrupt-stack-trace' into 'master'
Fix stack traces discovered with ^C forceful termination.
See merge request BuildStream/buildstream!1043
-
059035b9
by Tristan Van Berkom
at 2019-01-07T18:02:00Z
_scheduler/scheduler.py: Make _schedule_jobs() private
This is not used anywhere outside of the Scheduler, currently
only the Scheduler itself is allowed to queue a job at this level.
If the highlevel business logic for automatic queueing of auxiliary
jobs moves to another location, we can make this public again.
-
b83d1b1f
by Tristan Van Berkom
at 2019-01-07T18:02:00Z
_scheduler/scheduler.py: Only run one cache size job at a time
When queuing the special cache management related cleanup and
cache size jobs, we now treat these jobs as special and do the
following:
* Avoid queueing a cleanup/cache_size job if one is already queued
We just drop redundantly queued jobs here.
* Ensure that jobs of this type only run one at a time
This could have been done with the Resources mechanics,
however as these special jobs have the same properties and
are basically owned by the Scheduler, it seemed more straight
forward to handle the behaviors of these special jobs together.
This fixes issue #753
-
16a8816f
by Tristan Van Berkom
at 2019-01-07T18:02:00Z
Scheduler: Introduced JobStatus instead of simple success boolean
This changes the deepest callback from when a Job completes to
propagate a JobStatus value instead of a simple boolean, and updates
all of the effected code paths which used to receive a boolean
to now handle the JobStatus values.
This further improves the situation for issue #753, as now we avoid
queueing cache size jobs for pull jobs which are skipped.
-
c2fc2a5e
by Tristan Van Berkom
at 2019-01-07T18:02:00Z
_scheduler/jobs/job.py: Removed 'skipped' property
This is redundant now that we report it through the JobStatus.
-
3e3984ad
by Tristan Van Berkom
at 2019-01-07T18:50:23Z
Merge branch 'tristan/one-cache-size-job' into 'master'
Only queue one cache size job
Closes #753
See merge request BuildStream/buildstream!1040
-
512c726e
by Tristan Van Berkom
at 2019-01-08T03:38:11Z
sandbox/sandbox.py: Fix regression of command logging
Since we added batch commands, the batch commands print the
text of the commands directly in the message text, but this is wrong.
The detail string is the appropriate place for text of unknown lengths
(the user can actually configure how many max lines of commands they
want to see in their log), the message text itself should be controlled
and brief enough to avoid text wrapping.
-
01171988
by Tristan Van Berkom
at 2019-01-08T04:20:14Z
Merge branch 'tristan/fix-command-status-messages' into 'master'
sandbox/sandbox.py: Fix regression of command logging
See merge request BuildStream/buildstream!1044
-
6c1d06d6
by Phil Dawson
at 2019-01-08T10:24:32Z
element.py: remove reference to source bundle command
This command has been replacved by the bst source checkout command
-
914ecb72
by Jürg Billeter
at 2019-01-08T10:54:02Z
Merge branch 'phil/remove-source-bundle-reference' into 'master'
element.py: remove documentation reference to source bundle command
See merge request BuildStream/buildstream!1041
-
81f7a964
by Tristan Van Berkom
at 2019-01-08T16:55:58Z
sandbox/sandbox.py: Use assertions for programming errors instead of BstErrors.
When code is faulty, in a plugin or in the core, we should
get a stack trace and a BUG message.
-
edcc43ed
by Tristan Van Berkom
at 2019-01-08T19:19:10Z
Merge branch 'tristan/sandbox-programming-error' into 'master'
sandbox/sandbox.py: Use assertions for programming errors instead of BstErrors.
See merge request BuildStream/buildstream!1046
-
17c5ca2d
by Valentin David
at 2019-01-09T10:45:23Z
Use relative path to project directory for remote execution certificates/keys
-
605f8d11
by Valentin David
at 2019-01-09T10:45:23Z
Add support for user remote execution configuration
Fixes #631.
-
65ea03ab
by Valentin David
at 2019-01-09T10:45:23Z
Add support for https channel to remote execution and actions servers
Fixes #780.
-
ba05c4fa
by Valentin David
at 2019-01-09T12:58:59Z
Merge branch 'valentindavid/remote_execution_configuration' into 'master'
Remote execution configuration: HTTPS and user configuration
Closes #780 and #631
See merge request BuildStream/buildstream!1030
-
eb0dbcfc
by Valentin David
at 2019-01-09T13:56:58Z
buildstream/plugins/elements/script.py: Mark script as BST_VIRTUAL_DIRECTORY
ScriptElement does not use Sandbox.get_directory. It works using it
with remote execution.
Fixes #850
-
f7681925
by Valentin David
at 2019-01-09T14:44:22Z
Merge branch 'valentindavid/script_virtual_directory' into 'master'
buildstream/plugins/elements/script.py: Mark script as BST_VIRTUAL_DIRECTORY
Closes #850
See merge request BuildStream/buildstream!1047
-
f87d1c93
by Jim MacArthur
at 2019-01-09T15:01:18Z
sandboxremote.py: Rename 'instance_name' option to 'instance-name'
It was 'instance-name' in the documentation.
-
ed8bc5bc
by Jim MacArthur
at 2019-01-09T15:30:28Z
Merge branch 'jmac/rename_instance_name' into 'master'
Rename 'instance_name' option to 'instance-name'
See merge request BuildStream/buildstream!1048
-
f29a0995
by Tristan Van Berkom
at 2019-01-09T19:51:14Z
Support running test environments in parallel with `detox`
This patch namespaces the test temp directory and the output
coverage report file with the name of the environment under test,
such that separately run tests do not access the same files.
When running tests without tox, directly through setup.py,
then the tmp directory will still be `./tmp`.
* .gitignore: Added new .coverage-reports/ directory
* .gitlab-ci.yml: Rely on tox to combine and report coverage, only
tell tox about the COVERAGE_PREFIX so that results can be namespaced
by CI job name.
This change also publishes the sources and final combined `.coverage`
file in an output gitlab artifact for inspection, and lists some missing
dependencies to the `coverage` job.
* tox.ini: Add comments and refactor main [testenv] section so that
other environments dont inherit too much unrelated cruft.
Generate the coverate reports in the respective {envtmpdir} so that
all per-process coverage files are prefixed with a full path, ensuring
that concurrent runs don't mix reports and addressing concerns
raised in #844.
Also implemented new `tox -e coverage` environment to combine
any found coverage and print a report.
* .coveragerc: Omit .tox/ directory from coverage stats
Fixes issue #844
-
d364ad02
by Tristan Van Berkom
at 2019-01-09T19:52:22Z
CONTRIBUTING.rst: Updated to mention new `coverage` tox environment.
Also point out that it is possible to run test environments in
parallel using the `detox` tool.
-
1e352434
by Tristan Van Berkom
at 2019-01-09T20:36:06Z
Merge branch 'tristan/detox-tests' into 'master'
Allow using detox, and fix/refactor collection of coverage reports
Closes #844
See merge request BuildStream/buildstream!1051
-
116c1070
by Tristan Van Berkom
at 2019-01-09T20:39:15Z
.coveragerc: Omit some things which are irrelevant to cover.
* Omit versioneer's _version.py
* Omit our __main__.py which is used only internally for generating
documentation
-
c8f3616d
by Tristan Van Berkom
at 2019-01-09T21:12:29Z
Merge branch 'tristan/refine-coverage' into 'master'
.coveragerc: Omit some things which are irrelevant to cover.
See merge request BuildStream/buildstream!1052
-
c87bb592
by Jürg Billeter
at 2019-01-10T09:43:37Z
_platform/platform.py: Add canonicalize_arch() method
-
26e33346
by Jürg Billeter
at 2019-01-10T12:50:15Z
_options/optionarch.py: Accept architecture aliases
Accept common architecture aliases for arch options instead of only
accepting the canonicalized, OS-independent architecture name. This
restores compatibility with existing projects and may simplify option
handling for projects that only target a single OS (and thus, do not
need OS-independent architecture names).
-
06deb4c4
by Jürg Billeter
at 2019-01-10T12:50:15Z
element.py: Accept architecture aliases for sandbox config
Accept common architecture aliases for the sandbox config for
consistency with arch options.
-
0c2a66b3
by Jürg Billeter
at 2019-01-10T13:06:02Z
tests/format/optionarch.py: Add tests for architecture aliases
-
f86bc760
by Jürg Billeter
at 2019-01-10T13:34:56Z
Merge branch 'juerg/arch' into 'master'
Accept architecture aliases
See merge request BuildStream/buildstream!1034
-
347eb34e
by Angelos Evripiotis
at 2019-01-10T14:38:23Z
contributing: fix 'oprtation' and some other typos
-
6bc27bc1
by Angelos Evripiotis
at 2019-01-10T15:07:46Z
Merge branch 'aevri/contrib-typos' into 'master'
contributing: fix 'oprtation' and some other typos
See merge request BuildStream/buildstream!1055
-
0b83d024
by Tristan Van Berkom
at 2019-01-10T20:02:50Z
_pipeline.py: Fix the planner to retain the original order when depth sorting
The algorithm adds elements to a dictionary and then sorts the dictionary
by the discovered depths while recursing - using an OrderedDict is enough
to ensure a stable order.
This fixes `bst show --deps plan ...` reporting different results on
each invocation.
This fixes an aspect of #824
-
630e26f1
by Tristan Van Berkom
at 2019-01-10T20:03:52Z
testutils/runcli.py: Added Result.get_start_order()
Added a function to report the list of elements which appeared in
a given queue in the order of their first appearance.
-
d1d7de57
by Tristan Van Berkom
at 2019-01-11T04:13:16Z
tests/frontend/order.py: Adding tests ensuring stable element processing order
For each sample project and expected result order, this test ensures that:
* bst show --deps plan: Always shows the expected build order.
* bst source fetch: Always executes in the expected build order.
* bst build: Always builds in the expected order (disabled).
The build part of the test is currently disabled as the scheduler
seems to be messing with the order even in a `--builders 1` scenario.
-
276b8d48
by Tristan Van Berkom
at 2019-01-11T04:42:24Z
Merge branch 'tristan/element-processing-order' into 'master'
Make build plan element list stable
See merge request BuildStream/buildstream!1058
-
fc3dcec8
by Angelos Evripiotis
at 2019-01-11T16:45:00Z
dev-requirements: pytest-cov==2.6.1 for '--no-cov'
Bump the version of pytest-cov, so that we pick up the fix for
the '--no-cov' option breaking in pytest 4.0:
https://github.com/pytest-dev/pytest-cov/pull/230
This was the error you would get prior to this version:
_pytest.warning_types.RemovedInPytest4Warning: config.warn has been
deprecated, use warnings.warn instead
Although it says 'deprecated', it did result in
termination with stack trace.
-
f86b7ff3
by Angelos Evripiotis
at 2019-01-11T16:45:00Z
contributing: '--last-failed' and '--no-cov' tips
To help new folks get to grips with pytest, add some tips on other
frequently used features.
-
d983f231
by Angelos Evripiotis
at 2019-01-11T17:19:06Z
Merge branch 'aevri/tox_no_cov' into 'master'
pytest-cov==2.6.1, contributing: '--last-failed' and '--no-cov' tips
See merge request BuildStream/buildstream!1059
-
829a2b93
by Tristan Van Berkom
at 2019-01-12T21:32:13Z
_scheduler/queues/queue.py: Put elements in the skip list, not jobs
Minor correction, looks like we're not observing this queue
otherwise we'd be seeing crashes.
-
993e30ae
by Tristan Van Berkom
at 2019-01-12T22:02:55Z
Merge branch 'tristan/fix-skipped-elements' into 'master'
_scheduler/queues/queue.py: Put elements in the skip list, not jobs
See merge request BuildStream/buildstream!1063
-
32732e01
by Javier Jardón
at 2019-01-14T09:04:01Z
.gitlab-ci.yml: Use latest freedesktop-sdk 18.08.25
-
4f5f1184
by Valentin David
at 2019-01-14T09:40:56Z
Merge branch 'jjardon/freedesktop-sdk_latest' into 'master'
.gitlab-ci.yml: Use latest freedesktop-sdk 18.08.25
Closes #858
See merge request BuildStream/buildstream!1064
-
bb80a2b8
by Chandan Singh
at 2019-01-14T14:30:15Z
.gitlab-ci.yml: Use newer image for overning aarch tests
During recent reorganizing of `.gitlab-ci.yml` to work with `tox`, seems
like we missed to update the image used by the overnight aarch tests,
meaning that they currently fail due to `tox` being missing from them.
While these tests will be skipped on MRs usually, here is an example of
what the tests look like if they are actually run with this change:
https://gitlab.com/BuildStream/buildstream/-/jobs/145449561.
Fixes #859.
-
10b3ee62
by Chandan Singh
at 2019-01-14T14:57:08Z
Merge branch 'chandan/fix-overnight-aarch' into 'master'
Chandan/fix overnight aarch
Closes #859
See merge request BuildStream/buildstream!1065
-
542cdaf0
by Jürg Billeter
at 2019-01-14T18:46:57Z
_frontend/cli.py: Also check original args for --config in bash completion
The path of the config file generated by testutils for completion tests
is passed as regular argument, not via COMP_WORDS. Use that config file
in complete_artifact() to ensure the test uses the right artifact
directory.
-
ff666e76
by James Ennis
at 2019-01-14T18:46:57Z
completions.py: Add a test for our artifact ref autocompletions
Since the artifact subcommand group has been created, we have been able
to tab complete both element names and artifact refs as arguments to
the artifact subcommands (e.g. bst artifact log), this commit adds a
test for this.
-
273b0f55
by Tristan Van Berkom
at 2019-01-14T20:24:57Z
Merge branch 'jennis/add_artifacts_completion' into 'master'
completions.py: Add a test for our artifact ref autocompletions
See merge request BuildStream/buildstream!1054
-
74ac4237
by Chandan Singh
at 2019-01-15T00:23:20Z
.gitlab-ci.yml: Add job to attemp to update dependencies
This will help us check if BuildStream is working with the latest
version of dependencies, as per our constraints. This job is allowed to
fail but its failure should signal that we need to add stricter
constraints in some of our `.in` requirements files.
