[Notes] [Git][BuildStream/buildstream][jonathan/wsl-tests] 53 commits:

Jonathan Maw pushed to branch jonathan/wsl-tests at BuildStream / buildstream

Commits:

63b02f7c
by James Ennis at 2019-01-17T15:13:27Z
```
_profile.py: Add timestamp to the logs
```

f64ee46f

by James Ennis at 2019-01-17T17:02:28Z

_profile.py: Write binaries as well as logs

Private class methods which write the logs and write the binaries
have been added to Profile. The binaries are able to be used
by various visualisation tools.

2cb37a7e

by James Ennis at 2019-01-18T09:31:24Z

Merge branch 'jennis/profiling_outputs_binaries' into 'master'

Make the profiler output binaries (which can be used to visualise the data)

See merge request BuildStream/buildstream!1082

56ec33cc

by Valentin David at 2019-01-18T11:14:41Z

.gitlab-ci.yml: Add overnight tests logs to artifacts.

ad2df651

by Javier Jardón at 2019-01-18T12:01:56Z

Merge branch 'valentindavid/overnight-tests-logs-artifacts' into 'master'

.gitlab-ci.yml: Add overnight tests logs to artifacts.

See merge request BuildStream/buildstream!1084

f874295f

by Tristan Van Berkom at 2019-01-18T15:59:28Z

bzr source plugin: Use lock files to avoid corrupting the source cache

This patch by itself fixes #868

42a2fe3c

by Tristan Van Berkom at 2019-01-18T15:59:28Z

bzr source plugins: Remove all of the atomic backup directory business logic

Follow up of last commit which uses exclusive locking to
protect bzr operations instead.

a8713ed2

by Tristan Van Berkom at 2019-01-18T15:59:28Z

testutils/runcli.py: Added cli.get_element_states()

With get_element_state(), you need to invoke BuildStream once
for every element state you want to observe in a pipeline.

The new get_element_states() reports a dictionary with
the element state hashed by element name and is better to use
if you have more than one element to observe the state of.

a895cb2a

by Tristan Van Berkom at 2019-01-18T15:59:28Z

tests/frontend/track.py: Use test_track_recurse() to stress test Sources

This causes multiple source instances to interact with the same
backing data store at the same time, increasing the likelyhood
of triggering issues around concurrent access.

This more reliably triggers issue #868

4236bcc7

by Tristan Van Berkom at 2019-01-18T16:35:23Z

Merge branch 'tristan/fix-bzr-race' into 'master'

Fix bzr race conditions

Closes #868

See merge request BuildStream/buildstream!1083

951a8df1
by Angelos Evripiotis at 2019-01-18T18:02:22Z
```
cli.py: add a hint about '--' to 'bst shell' help
```

9911023f

by Angelos Evripiotis at 2019-01-18T18:56:21Z

Merge branch 'aevri/shell_separator_hint' into 'master'

cli.py: add a hint about '--' to 'bst shell' help

See merge request BuildStream/buildstream!1078

c9ce89d2

by Tristan Van Berkom at 2019-01-18T19:36:26Z

_cas/cascache.py: Cleanup directories when removing refs

With out this, empty directories in the refs/heads directory just
grow unconditionally.

c536ab6a

by Tristan Van Berkom at 2019-01-18T19:36:26Z

tests/artifactcache/expiry.py: Test refs directory cleanup

Enhance the test which checks removal of the extract directories
with an additional check that the ref directories are cleaned up
when removing artifacts.

99699ffc

by Tristan Van Berkom at 2019-01-18T19:36:26Z

utils.py: Added _tempnamedfile()

When used in a child process in BuildStream, this should be
used instead of tempfile.NamedTemporaryFile() directly, otherwise
we fail to cleanup the file on SIGTERM.

8ce483d4

by Tristan Van Berkom at 2019-01-18T19:36:26Z

_cas/cascache.py: Use utils._tempdir() and utils._tempnamedfile()

The direct usage of tempfile.TemporaryDirectory() and
tempfile.NamedTemporaryFile() here causes leakage of any temporary
data when the process operating on temporary data is terminated
with SIGTERM.

Using the utilities ensures that trash is not left behind in
~/.cache/buildstream/artifacts/tmp when the user terminates
BuildStream with ^C.

73c7252d

by Tristan Van Berkom at 2019-01-18T20:57:42Z

Merge branch 'tristan/cas-cleanup-improve' into 'master'

CASCache cleanup improvements

See merge request BuildStream/buildstream!1087

adfb9291

by Tristan Van Berkom at 2019-01-18T21:23:05Z

tests/testutils/python_repo.py: Use subprocess to run sdist

The current approach using setuptools.sandbox.run_setup() was
causing a spurious (but highly frequent) failure where setuptools
gets mixed up with it's manipulation of sys.modules and hits a
RuntimeError as a dictionary changes size while being iterated over.

For instance: https://gitlab.com/BuildStream/buildstream/-/jobs/147967307

Since this already happens in an isolated virtual environment created
by tox, we should not need additional sandboxing here from setuptools,
and launching this as a subprocess will be safer.

d114a6bd

by Tristan Van Berkom at 2019-01-18T22:07:22Z

Merge branch 'tristan/fix-pip-source-test' into 'master'

tests/testutils/python_repo.py: Use subprocess to run sdist

See merge request BuildStream/buildstream!1090

bf591ade

by Chandan Singh at 2019-01-18T22:09:04Z

Derive import plugin from Element instead of BuildElement

The `import` element is not really a build element. The main purpose of
the `BuildElement` class is to allow users to run `build-commands`,
`install-commands` etc. But, `import` does not run such commands.
Moreover, we already override all the methods provided by
`BuildElement`. So it only makes it confusing to have it derived from
`BuildElement` class when it is not a build element.

So, derive it from the base `Element` class instead.

2233a532

by Chandan Singh at 2019-01-18T22:46:41Z

Merge branch 'chandan/import-is-not-buildelement' into 'master'

Derive import plugin from Element instead of BuildElement

See merge request BuildStream/buildstream!1089

aa3411f9

by Tristan Maat at 2019-01-18T22:47:46Z

testutils/runcli.py: Allow removing artifacts from arbitrary dirs

`remove_artifact_from_cache` used a hard-coded path to remove
artifacts, which wasn't sufficient for integration tests.

fff882fe

by Tristan Maat at 2019-01-18T22:47:46Z

widget.py: Avoid "showing 0 lines" messages when there are no lines

This happened when bst is invoked with --message-lines 0 or
--error-lines 0, and was arguably a little too verbose (the user
explicitly asked us not to show them any lines, after all).

Fixes #779

e230dedb

by Tristan Van Berkom at 2019-01-18T23:23:35Z

Merge branch 'tlater/message-lines' into 'master'

Avoid "showing 0 lines" messages when we're asked to show no lines

Closes #779

See merge request BuildStream/buildstream!1031

43797617

by Richard Maw at 2019-01-21T10:41:37Z

_gitsourcebase.py: Fetch with explicit refspecs

Old versions of git lack --force and --tags,
but the same effect can be had by specifying refspecs.

ce8dab0f

by Richard Maw at 2019-01-21T10:41:37Z

tests/frontend/workspace.py: Skip test_open_multi_unwritable when root

The test assumes that a directory with write permission removed isn't writable,
this isn't the case if the process running the tests has CAP_DAC_OVERRIDE
which is common when running as root.

50165081

by Richard Maw at 2019-01-21T10:41:37Z

tests/sources/git.py: Skip tests that assume too new a git

test_track_invalid_submodule depends on being able to remove a submodule
by `git rm $submoduledir`, but old versions of git don't update .gitmodules
so BuildStream still thinks there's a submodule present.

For expediency the test is skipped rather than changed to manually remove
the entry from .gitmodules if git hasn't done it,
since in the common case git is new enough to do that itself.

test_git_describe expects --first-parent to find another tag,
but `bst track` will gracefully degrade if the option doesn't work
so a different history will be retained with old versions of git.

It's of marginal benefit to add additional cruft
to test for different output on old versions of git that won't persist forever.

8677a256

by Benjamin Schubert at 2019-01-21T10:41:37Z

Add a Centos runner

Centos is apparently different enough from fedora when running tests.

33782865

by Valentin David at 2019-01-21T11:47:18Z

Merge branch 'richardmaw/centos-oldgit-test-fixes' into 'master'

Fix CentOS

Closes #833

See merge request BuildStream/buildstream!1085

c00b3782

by Chandan Singh at 2019-01-21T19:37:53Z

CONTRIBUTING.rst: Fix formatting of link to pip docs

The link explaining python's requirements files was formatted
incorrectly (missing underscore at the end). This resulted in the text
being rendered literally, as opposed to being converted into a
hyperlink.

638ceb8a

by Chandan Singh at 2019-01-21T20:16:38Z

Merge branch 'chandan/fix-contrib-link' into 'master'

CONTRIBUTING.rst: Fix formatting of link to pip docs

See merge request BuildStream/buildstream!1094

383142e3