[Notes] [Git][BuildStream/buildstream][adamjones/systemd-cas] 81 commits

Adam Jones pushed to branch adamjones/systemd-cas at BuildStream / buildstream

Commits:

cb8c99e5

by Tristan Maat at 2018-07-29T07:52:06Z

buildelement.py: Only display one activity message (#507)

08b7562b

by Tristan Van Berkom at 2018-07-29T10:23:43Z

Merge branch '507-some-log-lines-appear-to-be-duplicates' into 'master'

Resolve "Some log lines appear to be duplicates"

Closes #507

See merge request BuildStream/buildstream!573

3b7158b3

by Tristan Van Berkom at 2018-07-29T10:25:02Z

source.py: Bring down since version for source mirroring APIs

Since we have now backported this to `bst-1.2`, the APIs have
been introduced in 1.2 and not 1.4

e84f8b24

by Tristan Van Berkom at 2018-07-29T10:25:02Z

NEWS: Bring down since version of backported features to 1.1.5

These features will be released in 1.1.5 since they have been
backported, lets reflect this in the NEWS file in master.

Also this adds the backported `remote` plugin to the NEWS

df18d38c

by Tristan Van Berkom at 2018-07-29T10:26:08Z

git.py: Handle concurrent download completions properly

Use os.rename() to rename the cloned temporary repository into
place in the source cache, and issue a STATUS message when discarding
a duplicate clone, in the case where the same repository is cloned
twice in parallel.

The problem with using shutil.move() is that it will create the source
directory in a subdirectory of the destination when the destination
exists, so it's behavior depends on whether the destination exists.

This shutil.move() behavior has so far hidden the race condition
where a duplicate repo is created in a subdirectory, as you need
to have three concurrent downloads of the same repo in order to
trigger the error.

This fixes issue #503

92714b03

by Tristan Van Berkom at 2018-07-29T11:18:52Z

Merge branch 'tristan/fix-spurious-errors' into 'master'

git.py: Handle concurrent download completions properly

Closes #503

See merge request BuildStream/buildstream!583

8568d272

by Phil Dawson at 2018-07-30T07:58:07Z

examples: add project demonstrating a junction element

This is part of the work towards #437.

efa24e7e

by Phil Dawson at 2018-07-30T07:58:07Z

junctions.run: Add terminal session output for junctions walkthrough.

This is part of the work towards #437.

bf175bf7

by Phil Dawson at 2018-07-30T07:58:07Z

junction-elements.rst: Add junctions walkthrough

This is part of the work towards #437.

e8931048
by James Ennis at 2018-07-30T07:58:07Z
```
using_advanced_features.rst: Added file
```

b4f25190

by Phil Dawson at 2018-07-30T07:58:07Z

tests/examples/junctions.py: Add tests for junctions walkthrough

This is part of the work towards issue #437.

aff76ae9

by Phil Dawson at 2018-07-30T09:15:49Z

Merge branch 'phil/437-junction-tutorial' into 'master'

Phil/437 junction tutorial

See merge request BuildStream/buildstream!550

c313ad17

by Tom Pollard at 2018-07-30T10:57:49Z

widget.py: Limit failure summary to currently failing elements

Change widget.py print_summary() to only print the failure
messages of elements in the Failure Summary that failed on the
current try.

4a3def5c

by Tristan Maat at 2018-07-30T11:47:21Z

Merge branch 'tpollard/386' into 'master'

widget.py: Limit failure summary to currently failing elements

See merge request BuildStream/buildstream!561

34015a26

by Tristan Maat at 2018-07-30T12:19:45Z

HACKING.rst: Add note about asking for dev permissions

ae47a72b

by Tristan Van Berkom at 2018-07-30T13:11:34Z

Merge branch 'tlater/ask-for-dev-permissions' into 'master'

HACKING.rst: Add note about asking for dev permissions

See merge request BuildStream/buildstream!587

695c0cb0

by Phil Dawson at 2018-07-30T16:07:36Z

junction-elements.rst: Add missing language specifier to literalinclude

A missing language specifier on a literal include directive results in yaml
being rendered as python code. This commit adds the :language: tag to that
directive.

fcb87b0a

by Phil Dawson at 2018-07-30T16:53:37Z

Merge branch 'phil/fixup-junctions-tutorial' into 'master'

junction-elements.rst: Add missing language specifier to literalinclude

See merge request BuildStream/buildstream!591

3d308894

by Richard Maw at 2018-07-31T16:31:36Z

element.py: Cache the result of checking whether an artifact is cached weakly

Normally we'd only need it in the case of scheduling a weakly cached build,
but to allow caching of failed builds we need to be able to distinguish
between cached successes and cached failures
for both strong and weak cache keys.

To allow other cache lookup codepaths to look up via the weak key
requires changes through the call stack to consult which key to use,
and cache invalidation of the saved state when it changes.

b3a68e28

by Richard Maw at 2018-07-31T16:31:36Z

element.py: Add metadata to distinguish between successful and failed builds

This just puts the metadata in place,
we're adding code paths to add failed builds later.

f4573df3

by Richard Maw at 2018-07-31T16:31:36Z

Convert call-sites of Element._cached() that assume success

When we later add cached failures it needs to not treat them as successes.

553df108

by Richard Maw at 2018-07-31T16:31:36Z

_frontend/widget.py: Render cached failures differently to successes

d14d8ee2

by Richard Maw at 2018-07-31T16:32:35Z

Cache failed builds

This creates an artifact when element assembly fails too,
and if it's the right kind of exception uses the now-included install directory
similarly to if it had returned successfully.

If there's a failure during install the artifact contains any installed files,
but may contain nothing at all.

d83122bb

by Richard Maw at 2018-07-31T16:32:35Z

_scheduler/queues/buildqueue.py: Skip rebuilding cached failures

This flags up a failure and if run in an interactive prompt
permits the user to attempt a rebuild.

78944e9a
by Richard Maw at 2018-07-31T16:32:35Z
```
tests: Add tests for cached behaviours
```

3fa79d8d

by Richard Maw at 2018-07-31T16:32:35Z

_scheduler/queues: Add failed builds to "done" queue

This allows the scheduler to move jobs from the current queue to the next.

As a result of this change later queues than the build queue
mustn't skip a cached failure, so the logic is specialised to build queues only.

e8cd43dc

by Richard Maw at 2018-07-31T16:32:58Z

NEWS: Describe caching of failures

Closes #76.

88cd61ea

by knownexus at 2018-07-31T16:32:58Z

Added the missing `_get_build_log` function

4fc1f5d1

by Phillip Smyth at 2018-07-31T17:17:34Z

Merge branch 'richardmaw/cache-fail' into 'master'

Store failed builds in the cache

Closes #76

See merge request BuildStream/buildstream!475

6cd3df09

by Phil Dawson at 2018-08-01T08:05:45Z

doc/sessions: Add terminal sessions for use in workspaces tutorial

This is part of the the work towards issue #437

8c0522ad

by Phil Dawson at 2018-08-01T08:05:45Z

Add project for workspaces tutorial

This is part of the work towards issue #437

dc60b713

by Phil Dawson at 2018-08-01T08:05:45Z

tutorials: Add workspaces tutorial

This is part of the work towards issue #437

bf8c3dfb

by Phil Dawson at 2018-08-01T08:05:45Z

testutils/patch.py: Add methods for applying and removing patches

b69783d4

by Phil Dawson at 2018-08-01T08:05:45Z

tests/examples/developing.py: Add tests for workspaces walkthrough

480565c2

by Phil Dawson at 2018-08-01T09:04:26Z

Merge branch 'phil/437-workspaces-tutorial' into 'master'

Phil/437 workspaces tutorial

Closes #437

See merge request BuildStream/buildstream!519

87edf67e

by Martin Blanchard at 2018-08-01T11:07:05Z

complete.py: Ensure paths get completed from 'element-path'

Element paths should always be completed from the root element folder
defined by the element-path key in project.conf. Fix complete_path() to
always search into its given base_directory argument.

See issue BuildStream/buildstream#448

d10e4668

by Phillip Smyth at 2018-08-01T12:05:40Z

Merge branch 'mablanch/448-autocompletion-broken-defaults' into 'master'

Fix autocompletion for elements in sub-folders

See merge request BuildStream/buildstream!592

6aedc57b

by Jim MacArthur at 2018-08-01T12:07:32Z

Add BST_VIRTUAL_DIRECTORY flag for element plugins

1d3af84d

by Jim MacArthur at 2018-08-01T12:07:32Z

utils.py: add getmtime() and magic_timestamp

magic_timestamp is moved into file scope so other classes
can use it.

4397e45a

by Jim MacArthur at 2018-08-01T12:07:32Z

Sandbox.py: Rename __root to _root.

This is to allow to allow its use by subclasses.

Since access to get_directories is now blocked for some plugins,
and the subclasses of Sandbox do not have configuration defined
by YAML files, they need another way to get at the root directory.

9bf343cf

by Jim MacArthur at 2018-08-01T12:07:32Z

Add the virtual directory class 'Directory' and one implementation.

buildstream/storage/Directory.py: New file.
buildstream/storage/_filebaseddirectory.py: New file.
buildstream/_exceptions.py: New VIRTUAL_FS exception source.

f124dfec
by Jim MacArthur at 2018-08-01T12:07:32Z
```
storage: Add __init__.py
```

42bdce0b

by Jim MacArthur at 2018-08-01T12:07:32Z

sandbox.py: Add function to get virtual directory.

fe1a3428