[Notes] [Git][BuildStream/buildstream][juerg/command-batching] 35 commit

Jürg Billeter pushed to branch juerg/command-batching at BuildStream / buildstream

Commits:

27932739

by Benjamin Schubert at 2018-11-19T09:46:39Z

plugins/sources/pip.py: also look for python version named "python"

In some virtual environments, we might not have the binary with the
exact python version, which would lead the current pip plugin to
fail to find a working pip version.

This adds "python" at the start of the list of valid python versions

ea2de561

by Benjamin Schubert at 2018-11-19T10:19:24Z

Merge branch 'bschubert/fix-pip-python' into 'master'

plugins/sources/pip.py: also look for python version named "python"

Closes #758

See merge request BuildStream/buildstream!942

f23b6031

by Benjamin Schubert at 2018-11-19T11:39:51Z

tests/frontend/buildtrack.py: check for success after invocation

Previously the result of the first call in test_build_track was not
checked to be successful, which would lead to potential erros later on
that were hard to debug.

Let's check it everytime

a6defc0b

by Benjamin Schubert at 2018-11-19T11:39:51Z

Fix os.rename in git source element to correctly handle error codes

According to the documentation
(https://www.unix.com/man-page/POSIX/3posix/rename/), when the directory
already is there, either EEXIST or ENOTEMPTY could be thrown.

Previously only ENOTEMPTY was checked.

Done:
  - Separated the move into its own function
  - Check for both errors
  - Create unit tests for it, covering most test cases

88089d2d

by Benjamin Schubert at 2018-11-19T11:39:51Z

Extract atomic move function to utils.py

Moving atomically a file/directory can be tricky since different
errors might be raised for the same underlying problem.

Having a utility function to reduce this discrepancies will help
in ensuring we have correct behavior

fd9e46be

by Benjamin Schubert at 2018-11-19T11:39:51Z

cascache.py: use move_atomic instead of manual error checking

d32e0b83

by Benjamin Schubert at 2018-11-19T11:39:51Z

sources/pip.py: Use move_atomic instead of manual os.rename

This uses move_atomic insteand of the manual os.rename and manual error
checking and throws a SourceError for consistency with other modules.

6f837118

by Benjamin Schubert at 2018-11-19T12:22:40Z

Merge branch 'bschubert/fix-atomic-move-git-repo' into 'master'

Fix os.rename in git source element to correctly handle error codes

See merge request BuildStream/buildstream!938

30b72244

by Chandan Singh at 2018-11-19T12:35:01Z

buildstream/_pipeline.py: Simplify error message for assert_consistent()

Currently, `Pipeline.assert_consistent()` prints an error message like
the one below when certain elements have inconsistent sources:

```
...
      Element: hello.bst is inconsistent
        Source tar source at hello.bst [line 16 column 2] is missing ref
...
```

Drop the word "Source" from the beginning of the message as
"Source <source-kind> source ..." is kind of awkward to read.

76c5d2f8

by Chandan Singh at 2018-11-19T12:35:01Z

Add `bst source-checkout` command

As discussed in
https://mail.gnome.org/archives/buildstream-list/2018-September/msg00064.html,
add `bst source-checkout` command. This will allow users to checkout
sources for a given target.

* _frontend/cli.py: Add source-checkout command
* _pipeline.py: Add assert_sources_cached() method
* _stream.py: Add source_checkout method, abstract out
  __check_location_writable() method that used to part of checkout()

9f629638

by Chandan Singh at 2018-11-19T13:03:36Z

Merge branch 'chandan/source-checkout' into 'master'

Add `bst source-checkout` command

See merge request BuildStream/buildstream!820

bc827cc8

by Abderrahim Kitouni at 2018-11-19T14:55:15Z

plugins/elements/cmake.yaml: always specify variable types

cmake sometimes misinterprets relative paths as relative to the current directory
if this is not specified. See freedesktop-sdk/freedesktop-sdk#431

adjust tests/format/variables.py accordingly.

5fbc5f41

by Valentin David at 2018-11-19T15:31:05Z

Merge branch 'abderrahim/cmake-variable-types' into 'master'

plugins/elements/cmake.yaml: always specify variable types

Closes #761

See merge request BuildStream/buildstream!947

06e28860

by Benjamin Schubert at 2018-11-19T15:52:24Z

Don't cache sandbox errors

Sandbox errors (like missing host tools) are dependent on the host
system and rarely on what is actually done.

It is therefore better to not cache them as they are subject to
change between two runs.

Also add test to ensure sandbox failure are not cached

dd36cfbc

by Benjamin Schubert at 2018-11-19T15:52:24Z

element.py: Only cache on element error or success

Reverse the way we were handling caching by only caching
when we know it's a problem with the build or the build is a
success in order not to cache transient errors.

eac7274d

by Benjamin Schubert at 2018-11-19T16:29:07Z

Merge branch 'bschubert/dont-cache-errors-from-host-tools' into 'master'

Don't cache sandbox failures

Closes #727

See merge request BuildStream/buildstream!895

625dfe1f

by Chandan Singh at 2018-11-19T16:41:17Z

NEWS: Add entry for the new source-checkout command

`bst source-checkout` command was recently added in
https://gitlab.com/BuildStream/buildstream/merge_requests/820. Add a
NEWS entry for the new command.

d4f12184

by Chandan Singh at 2018-11-19T17:15:10Z

Merge branch 'chandan/source-checkout-news' into 'master'

NEWS: Add entry for the new source-checkout command

See merge request BuildStream/buildstream!960

ba6c96e8

by Richard Maw at 2018-11-19T19:54:30Z

CasBasedDirectory: Change constructor to take a CASCache instead of a Context

The Context was only used to obtain a reference to the CASCache
and set the unused cas_directory field.

8a0dc3a3

by Jürg Billeter at 2018-11-19T19:54:30Z

tests/testutils/artifactshare.py: Do not create a fake context

The fake context did not set the cache quota, triggering an error. With
CASCache now separate from ArtifactCache, we can instantiate a CASCache
without context.

8722aced

by Jürg Billeter at 2018-11-19T19:54:30Z

tests/plugins/pipeline.py: Load context default values

fc56ffa4

by Jürg Billeter at 2018-11-19T19:54:30Z

_context.py: Drop duplicated default values for user configuration

The default values are in userconfig.yaml, together with the
documentation. The default values should not be duplicated in
_context.py.

3f663d82

by Jürg Billeter at 2018-11-19T20:36:57Z

Merge branch 'juerg/context-default-values' into 'master'

_context.py: Drop duplicated default values for user configuration

See merge request BuildStream/buildstream!953

ad6cde1b

by Jürg Billeter at 2018-11-20T06:33:50Z

sandbox: Deduplicate code to process cwd, env, and command arguments

c9a1e7a4

by Jürg Billeter at 2018-11-20T06:37:01Z

element.py: Pass element object to sandbox

This allows the sandbox to output messages using the plugin id.

e152a93d