[Notes] [Git][BuildStream/buildstream][willsalmon/outOfSourecBuild] 44 c

Will Salmon pushed to branch willsalmon/outOfSourecBuild at BuildStream / buildstream

Commits:

11320fe2

by Tristan Van Berkom at 2018-10-03T09:33:39Z

_scheduler: Fix bookkeeping of terminated jobs

 * Enhanced the base Job class to bookkeep which jobs have
   been terminated, and to consider them as `skipped` when
   asked via the `skipped` property.

 * Enhanced the base Queue class to bookkeep the job statuses
   more carefully.

This fixes #479

91271964

by Tristan Van Berkom at 2018-10-03T09:59:40Z

Merge branch 'tristan/fix-terminated-jobs' into 'master'

_scheduler: Fix bookkeeping of terminated jobs

Closes #479

See merge request BuildStream/buildstream!850

3bf895d2

by Jonathan Maw at 2018-10-03T11:48:48Z

tests: Add regression test for mirroring with project.refs

e4969807

by Jonathan Maw at 2018-10-03T12:48:07Z

Merge branch 'jonathan/source-mirror-project-refs' into 'master'

tests: Add regression test for mirroring with project.refs

See merge request BuildStream/buildstream!823

a0814399

by Tristan Van Berkom at 2018-10-03T13:05:52Z

_scheduler/scheduler.py: Ignore interrupt events while terminating.

For some reason, we now receive a SIGINT from the main loop even
when the SIGINT occurred with the handler disconnected in an interactive
prompt. This patch simply ignores any received SIGINT events from the
main loop in the case that we are already in the process of terminating.

This fixes issue #693

0a1f8e3c

by Tristan Van Berkom at 2018-10-03T13:42:20Z

Merge branch 'tristan/fix-double-terminate-prompt' into 'master'

_scheduler/scheduler.py: Ignore interrupt events while terminating.

Closes #693

See merge request BuildStream/buildstream!852

11161f99

by Tristan Van Berkom at 2018-10-03T13:44:02Z

utils.py: Give save_file_atomic() a tempdir argument

Allow callers to decide where the temporary file will be created.

3e797bb9

by Tristan Van Berkom at 2018-10-03T13:44:02Z

_artifactcache/cascache.py: Don't create temporary files in the CAS storage

Use the designated tempdir when creating refs, we expect that temporary
files are not created in the storage directory ever, they should be
only ever created in the designated temporary directory.

This fixes race conditions where utils._get_dir_size() throws an
unhandled exception when attempting to stat the file which inadvertantly
disappears.

d9020e43

by Tristan Van Berkom at 2018-10-03T13:44:02Z

utils.py: Document _get_dir_size() expectations.

This function assumes that files do not disappear while
walking the given directory.

3e5ff5a9

by Tristan Van Berkom at 2018-10-03T14:09:51Z

Merge branch 'tristan/fix-cache-size-race' into 'master'

fix cache size race

See merge request BuildStream/buildstream!854

e8179c34

by Jim MacArthur at 2018-10-03T16:06:59Z

element.py: Put the temporary staging directory in artifactdir

3cf38c8e

by Jim MacArthur at 2018-10-03T16:44:02Z

Merge branch 'jmac/temporaries-inside-cachedir' into 'master'

Move the temporary staging directory to artifactdir

See merge request BuildStream/buildstream!856

59c92bda

by Daniel Silverstone at 2018-10-04T08:12:20Z

sandbox/_sandboxdummy.py: Take a reason for use

Since the SandboxDummy sandbox is only used in very specific
circumstances and with good reason, this adjusts the SandboxDummy
class to take a reason and to return it if `.run()` has to
raise an exception because this is a dummy sandbox.

Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>

b9ddcd0e

by Daniel Silverstone at 2018-10-04T08:12:20Z

_platform/darwin.py: Give reason for use of dummy sandbox

Since Darwin is limited to the dummy sandbox for now due to OSXFUSE
being unsupported as yet, and there being no suitable sandboxing option
for use on OSX, give this as the reason so that it can be reported if
the user attempts a local build.

Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>

df0d5a8b

by Daniel Silverstone at 2018-10-04T09:04:21Z

_platform/linux.py: Refactor checks for sandboxing

To better report issues in the absence of a suitable bwrap, or the
FUSE devices, this refactors the checks for sandboxing in the Linux
platform to cover the various possibilities.  The reasons are then
collated and passed to the dummy sandbox for later reporting to the
user if a local build is attempted.

Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>

b8421a9c

by Daniel Silverstone at 2018-10-04T09:04:21Z

_site.py: Protect against failure running `bwrap --version`

On some potentially broken systems, running `bwrap --version` might
fail with an error code.  This patch corrects the oversight ensuring
that we cleanly return False for version checking in such cases.

Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>

c74bfbe5

by Daniel Silverstone at 2018-10-04T09:04:21Z

_site.py: Reduce complexity of bwrap version comparison

Instead of an if/else ladder which is quite complex, this patch
uses the fact that Python supports by-component tuple comparison
to simply compare two (major, minor, patch) tuples

Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>

c46a7e87

by Daniel Silverstone at 2018-10-04T09:13:04Z

setup.py: Change bwrap assertion to a warning

Since there are use-cases where BuildStream could be installed
onto systems which do not have BubbleWrap (e.g. for remote-build-only
scenarios) it is not correct to assert a dependency on bwrap during
installation.  This patch makes the assertion a warning, and also
clarifies the message somewhat.  This should fix #644

Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>

fd6a9573

by Jürg Billeter at 2018-10-04T09:45:58Z

Merge branch 'danielsilverstone-ct/bwrap-check-runtime-only' into 'master'

Make bwrap check runtime only

Closes #644

See merge request BuildStream/buildstream!847

c5778941

by Josh Smith at 2018-10-04T13:43:10Z

element.py: Do not save workspace on failed build

This fixes #634, BuildStream no longer saves workspace configuration on
a failed build.

788cde6a

by Josh Smith at 2018-10-04T13:43:10Z

testing: Add regression test for workspace bug

8630bac4

by Tristan Van Berkom at 2018-10-04T14:11:27Z

Merge branch 'Qinusty/634-workspace-failed-builds' into 'master'

Do not save workspace on failed build

Closes #634

See merge request BuildStream/buildstream!812

a7984218

by Valentin David at 2018-10-04T14:56:09Z

Catch correct exception from shutil.rmtree

Python documentation is not clear on what shutil.rmtree is supposed to
raise. However from the source code, it is obvious that OSError are
raised, but never shutil.Error.

It is not easy to test in normal condition because issues happen
usually in combination with a FUSE filesystem, a probably a race
condition where `fstatat` returns an error just before the filesystem
being unmounted.

Fixes #153.

b7b20d9e

by Valentin David at 2018-10-04T15:31:22Z

Merge branch 'valentindavid/rmtree_oserror' into 'master'

Catch correct exception from shutil.rmtree

Closes #153

See merge request BuildStream/buildstream!849

42ea143d

by Chandan Singh at 2018-10-05T12:40:48Z

Add contrib script to generate Docker images from bst checkout

This script can be useful to generate a Docker image from `bst checkout`
of an element as a single command. While this script does not eliminate
the need for a proper Docker/OCI element plugin that would probably also
support layering, it provides a cheap way to export the element into a
Docker image.

b38f9f9d

by Jonathan Maw at 2018-10-05T13:05:38Z

Merge branch 'chandan/bst-docker-import' into 'master'

Add contrib script to generate Docker images from bst checkout

See merge request BuildStream/buildstream!857

47df1e36

by Jürg Billeter at 2018-10-05T13:30:38Z

element.py: Prepare local sandbox for bst checkout and bst shell

A remote execution sandbox is not suitable for these commands.

d3921ab8

by Jonathan Maw at 2018-10-05T13:54:17Z

Merge branch 'juerg/remote' into 'master'

element.py: Prepare local sandbox for bst checkout and bst shell

See merge request BuildStream/buildstream!860

53d9f977

by Tristan Van Berkom at 2018-10-06T14:40:46Z

CONTRIBUTING.rst: Updating CONTRIBUTING guidelines

This is almost a complete rewrite of the CONTRIBUTING guide.

  * The patch submission guidelines have become less ambiguous and more strict

  * Some general restructuring and reordering of the file took place

  * The codeing guidelines have changed completely.

    o There is much less room for ambiguity here now
    o More emphasis on consistency in the codebase
    o Added some more abstract points which should be considered
      when writing and reviewing patches

  * The policy on public/private symbols has been greatly clarified

  * Added new section about adding new core plugins and what needs
    to be done as a consequence of that

5587715c

by Tristan Van Berkom at 2018-10-07T06:19:03Z

CONTRIBUTING.rst: Some fixes in grammer and minor corrections

43c7ac23
by Angelos Evripiotis at 2018-10-07T15:19:58Z
```
contributing.rst: fix typo
```

3f0c919f

by Angelos Evripiotis at 2018-10-07T15:52:52Z

Merge branch 'aevri/fixup-contributing-typos' into 'master'

contributing.rst: fix typos

See merge request BuildStream/buildstream!865

6f5f795e

by Tristan Van Berkom at 2018-10-08T08:46:11Z

CONTRIBUTING.rst: Added section about minimizing API surfaces

And modified some titles so that the titles in CONTRIBUTING.rst actually
follow the documentation guidelines for naming of section titles.

abb9fb98

by Tristan Van Berkom at 2018-10-08T08:59:18Z

CONTRIBUTING.rst: Added section on file naming convention

ba905e3a

by Tristan Van Berkom at 2018-10-08T09:05:18Z

CONTRIBUTING.rst: Correcting `Context` example in the "imports" section

This example is very outdated and comes from pre 1.0 when the Context
object was going to be public API. Just corrected the example made about
imports to import the `Context` object from `._context` instead of `.context`.

674afe94

by Tristan Van Berkom at 2018-10-08T09:07:45Z

CONTRIBUTING.rst: Added a note that public API objects should be exposed from __init__.py

350c6796

by Tristan Van Berkom at 2018-10-08T10:44:47Z

CONTRIBUTING.rst: Added section in PEP-8 coding style about line lengths.

We have a long line limit in order to handle the exceptions without making
code unreadable as a result, this long line length limit is not an invitation
to fill up the limit as much as possible.

73775df3