[Notes] [Git][BuildStream/buildstream][juerg/command-batching] 178 commi

Jürg Billeter pushed to branch juerg/command-batching at BuildStream / buildstream

Commits:

42ea143d

by Chandan Singh at 2018-10-05T12:40:48Z

Add contrib script to generate Docker images from bst checkout

This script can be useful to generate a Docker image from `bst checkout`
of an element as a single command. While this script does not eliminate
the need for a proper Docker/OCI element plugin that would probably also
support layering, it provides a cheap way to export the element into a
Docker image.

b38f9f9d

by Jonathan Maw at 2018-10-05T13:05:38Z

Merge branch 'chandan/bst-docker-import' into 'master'

Add contrib script to generate Docker images from bst checkout

See merge request BuildStream/buildstream!857

47df1e36

by Jürg Billeter at 2018-10-05T13:30:38Z

element.py: Prepare local sandbox for bst checkout and bst shell

A remote execution sandbox is not suitable for these commands.

d3921ab8

by Jonathan Maw at 2018-10-05T13:54:17Z

Merge branch 'juerg/remote' into 'master'

element.py: Prepare local sandbox for bst checkout and bst shell

See merge request BuildStream/buildstream!860

53d9f977

by Tristan Van Berkom at 2018-10-06T14:40:46Z

CONTRIBUTING.rst: Updating CONTRIBUTING guidelines

This is almost a complete rewrite of the CONTRIBUTING guide.

  * The patch submission guidelines have become less ambiguous and more strict

  * Some general restructuring and reordering of the file took place

  * The codeing guidelines have changed completely.

    o There is much less room for ambiguity here now
    o More emphasis on consistency in the codebase
    o Added some more abstract points which should be considered
      when writing and reviewing patches

  * The policy on public/private symbols has been greatly clarified

  * Added new section about adding new core plugins and what needs
    to be done as a consequence of that

5587715c

by Tristan Van Berkom at 2018-10-07T06:19:03Z

CONTRIBUTING.rst: Some fixes in grammer and minor corrections

43c7ac23
by Angelos Evripiotis at 2018-10-07T15:19:58Z
```
contributing.rst: fix typo
```

3f0c919f

by Angelos Evripiotis at 2018-10-07T15:52:52Z

Merge branch 'aevri/fixup-contributing-typos' into 'master'

contributing.rst: fix typos

See merge request BuildStream/buildstream!865

6f5f795e

by Tristan Van Berkom at 2018-10-08T08:46:11Z

CONTRIBUTING.rst: Added section about minimizing API surfaces

And modified some titles so that the titles in CONTRIBUTING.rst actually
follow the documentation guidelines for naming of section titles.

abb9fb98

by Tristan Van Berkom at 2018-10-08T08:59:18Z

CONTRIBUTING.rst: Added section on file naming convention

ba905e3a

by Tristan Van Berkom at 2018-10-08T09:05:18Z

CONTRIBUTING.rst: Correcting `Context` example in the "imports" section

This example is very outdated and comes from pre 1.0 when the Context
object was going to be public API. Just corrected the example made about
imports to import the `Context` object from `._context` instead of `.context`.

674afe94

by Tristan Van Berkom at 2018-10-08T09:07:45Z

CONTRIBUTING.rst: Added a note that public API objects should be exposed from __init__.py

350c6796

by Tristan Van Berkom at 2018-10-08T10:44:47Z

CONTRIBUTING.rst: Added section in PEP-8 coding style about line lengths.

We have a long line limit in order to handle the exceptions without making
code unreadable as a result, this long line length limit is not an invitation
to fill up the limit as much as possible.

3ca487b8

by Chandan Singh at 2018-10-08T18:25:39Z

bst-docker-import: Consistently use stderr for all logs

In !857, we added `contrib/bst-docker-import` script but it has a small
issue that some of logs go to stdout while others go to stderr. Fix it
so that all logging is done on stderr.

49df3d75

by Chandan Singh at 2018-10-08T18:58:54Z

Merge branch 'chandan/fix-bst-docker-import-logging' into 'master'

bst-docker-import: Consistently use stderr for all logs

See merge request BuildStream/buildstream!863

d827dfae
by Angelos Evripiotis at 2018-10-09T09:12:45Z
```
contributing.rst: end lines with punctuation
```

9590e8ae

by Angelos Evripiotis at 2018-10-09T09:12:45Z

contributing.rst: fix whitespace

No tabs, no lines of only horizontal whitespace.

6d02e269
by Angelos Evripiotis at 2018-10-09T09:12:45Z
```
contributing.rst: fix typo of 'get_count'
```

3ed26a47

by Angelos Evripiotis at 2018-10-09T09:12:45Z

contributing.rst: de-paren subclass example

It seems easier to read when separated into two sentences.

5b22d850
by Angelos Evripiotis at 2018-10-09T09:12:45Z
```
contributing.rst: no spaces before '?'
```

ac0776f8

by Tristan Van Berkom at 2018-10-09T09:36:33Z

Merge branch 'aevri/contributing_fixups' into 'master'

Minor fixups to contributing.rst

See merge request BuildStream/buildstream!866

885bd946

by Tristan Van Berkom at 2018-10-09T09:39:24Z

CONTRIBUTING.rst: Added note about sphinx supporting docstrings on instance variables

Python does not natively support this, but sphinx does parse them and includes
these in the generated documentation.

120d8c73

by Tristan Van Berkom at 2018-10-09T09:45:23Z

CONTRIBUTING.rst: Fix typos in previous commit.

Ooops, that was done far too quickly.

cda1354c

by Chandan Singh at 2018-10-10T00:17:51Z

doc: Add new document about Docker integrations

Add a new section under "Additional writings" section about how
BuildStream integrates with Docker. At present, this section includes
only two sections:

- "Run BuildStream inside Docker", that refers to existing instructions
for doing so
- "Generate Docker images", that adds instructions for using the newly
added `contrib/bst-docker-import` script

In future, this document can be extended to include the following
things:

- Importing Docker sources via `docker` source plugin - if/when the
Docker source plguin is moved from bst-external to BuildStream proper
- A Docker sandbox, if/when that is added

dd472d95

by Chandan Singh at 2018-10-10T00:18:17Z

NEWS: Add entry about contrib/bst-docker-import

As suggested in
https://gitlab.com/BuildStream/buildstream/merge_requests/857#note_106809743,
add a NEWS entry for `contrib/bst-docker-import` that was added
recently.

35ac26a7

by Chandan Singh at 2018-10-10T00:59:17Z

Merge branch 'chandan/bst-and-docker' into 'master'

Add documentation and NEWS entry about bst-docker-import

See merge request BuildStream/buildstream!864

7d96333f
by Laurence Urhegyi at 2018-10-10T19:24:14Z
```
README: Update to add link to website
```

39492db8

by devcurmudgeon at 2018-10-10T19:24:14Z

Merge branch 'patch-2' into 'master'

README: Update to add link to website

See merge request BuildStream/buildstream!859

4f0bfb4a

by Tristan Van Berkom at 2018-10-11T15:37:51Z

Rename element_enums.py -> types.py

This will be the place to store low level data types used
throughout the core, for now this includes public and private
types.

9dc10cc7

by Tristan Van Berkom at 2018-10-11T15:45:36Z

doc/source/core_framework.rst: Include the foundation types in the API docs.

efc0d4cc

by Tristan Van Berkom at 2018-10-11T15:48:44Z

types.py: Moved Consistency definition here from source.py

And slightly touch up the doc strings for Consistency and Scope.

a0712ead

by Tristan Van Berkom at 2018-10-11T16:23:54Z

Merge branch 'tristan/refactor-types-api' into 'master'

Refactor types api

See merge request BuildStream/buildstream!870

ce55b9a0

by Tiago Gomes at 2018-10-12T10:15:46Z

doc: updates considering website being live now

* Add a link to the website on the main page.
* Remove install instructions as they are now on the website.
* Remove Resources section as that information can be found at the
  website, and also looks bad.
* Move artifact server setup from the no longer existing Install section
  to the Using section.

10b092e1

by Tristan Van Berkom at 2018-10-12T10:21:27Z

doc/source/additional_docker.rst: Fix link to refer to website

The docker guide, which is part of the install guide, has moved
to the website.

1c7ba9a0

by Tristan Van Berkom at 2018-10-12T10:24:31Z

doc/source/index.rst: Moved references to the website below the simple ToC.

3738dd06

by Tristan Van Berkom at 2018-10-12T10:49:39Z

Merge branch 'tristan/remove-install-guide' into 'master'

Remove install guide

See merge request BuildStream/buildstream!872

c31ed138

by Tristan Van Berkom at 2018-10-14T15:24:53Z

CONTRIBUTING.rst: Added more guidelines about documenting the user guide

b39d87b4

by Jürg Billeter at 2018-10-15T11:07:05Z

_artifactcache: Fix crash in push_directory() without artifact server

Fixes #708.

4e3ec89e

by Jürg Billeter at 2018-10-15T11:48:16Z

Merge branch 'juerg/remote-config' into 'master'

_artifactcache: Fix crash in push_directory() without artifact server

Closes #708

See merge request BuildStream/buildstream!871

d15ed8fc

by Jonathan Maw at 2018-10-15T15:54:59Z

yaml: Add a cache of parsed and provenanced yaml

Note that the ProvenanceFile's names will be incorrect after loading
from the cache, but this is currently only used for writeback, which
isn't used in junctions.

f4eb685a
by Jonathan Maw at 2018-10-15T15:54:59Z
```
yamlcache.py: Add yaml cache tests
```

26dfa882

by Lachlan Mackenzie at 2018-10-15T15:54:59Z

yaml.py: Add loader for yaml test files cache option

* Fix to CWD issue provided by Jonathan Maw

0fbff08b

by Lachlan Mackenzie at 2018-10-15T15:54:59Z

yaml.py: Add yaml cache testing to yaml list composition test

3d58d7c5

by Lachlan Mackenzie at 2018-10-15T15:54:59Z

yaml.py: Add yaml cache test to yaml list composition twice

363cfc39

by Lachlan Mackenzie at 2018-10-15T15:54:59Z

yamlcache.py: Add YAML cache changed file test

* Test in same style as test_yamlcache_used
* Move of project not required so removed

086c47c4

by Jonathan Maw at 2018-10-15T16:19:01Z

Merge branch 'lachlan/pickle-yaml-test-list-composite' into 'master'

Add YAM cache testing to yaml tests and further YAML cache refinements

See merge request BuildStream/buildstream!836

6298e409

by William Salmon at 2018-10-16T12:39:14Z

Sandbox: CWD was not being created for workspaces

The code was creating the cwd folder but when the workspace was
mounted in to the buildroot it was hiding the folder created in it
behind the bind mounted workspace.

However by using the bubblewarp `--dir` directive to ensure that cwd
exists we can cover both workspace and non workspace situations with
the same method.

For issue #512 in Gitlab.

bbcb9bd5

by William Salmon at 2018-10-16T12:39:14Z

New Integration test for workspace command-subdir

Build stream was only creating command-subdir for non workspace builds
this has now been fixed, and this regression test should support the
feature.

For issue #512 in Gitlab.

5e7cc645

by William Salmon at 2018-10-16T12:39:14Z

Add conf-root variable to builds

Adding the conf-root variable makes creating out of source builds
slightly easier.

For issue #512 in Gitlab.

0d04e1b7

by William Salmon at 2018-10-16T12:39:14Z

Added tests for conf-root variable

For issue #512 in Gitlab.

b2767ac0

by William Salmon at 2018-10-16T12:39:14Z

Extended the Documentation to cover out of source builds

For issue #512 in Gitlab.

118704a9

by William Salmon at 2018-10-16T12:39:14Z

Added a news item for building out of the source directory

For issue #512 in Gitlab.

0a5db229

by Phil Dawson at 2018-10-16T13:19:56Z

Merge branch 'willsalmon/outOfSourecBuild' into 'master'

Out of source builds

Closes #512

See merge request BuildStream/buildstream!776

1fd38fb7

by Jim MacArthur at 2018-10-16T14:54:09Z

bst2html.py: Name the second argument to avoid an error

b023d5c0

by Jim MacArthur at 2018-10-16T15:28:15Z

Merge branch 'jmac/doc-argument-fix' into 'master'

Add named argument in bst2html.py to fix documentation

See merge request BuildStream/buildstream!874

ac5b482d
by Richard Dale at 2018-10-16T19:40:39Z
```
Fix spelling of it's and its
```

3d9c4eca

by Jürg Billeter at 2018-10-16T20:17:39Z

Merge branch 'rdale/spelling' into 'master'

Fix spelling of it's and its

See merge request BuildStream/buildstream!877

edc07536

by Tiago Gomes at 2018-10-17T13:49:15Z

README: use correct links for documentation website

155296a7

by Tiago Gomes at 2018-10-17T13:49:15Z

setup.py: use correct link for documentation website

131f95eb

by Tiago Gomes at 2018-10-17T14:15:38Z

Merge branch 'tiagogones/update-links-to-documentation' into 'master'

Update links to documentation website

See merge request BuildStream/buildstream!876

c327d1f3
by Valentin David at 2018-10-17T14:33:40Z
```
Change URL to the Alpine tarball
```

eeb334f2

by Javier Jardón at 2018-10-17T15:03:17Z

Merge branch 'valentindavid/move_alpine_tarball' into 'master'

Change URL to the Alpine tarball

See merge request BuildStream/buildstream!880

b8f69946

by Daniel Silverstone at 2018-10-18T09:32:09Z

_loader/loader.py: Be more explanatory in CircDep exception

When a circular dependency is detected, the reported error hilights
only one part of the loop in question.  This change ensures that
the entire loop is listed so that the user has a greater chance of
easily determining the correct fix.

Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>

892866e1

by richardmaw-codethink at 2018-10-18T10:04:32Z

Merge branch 'danielsilverstone-ct/report-loop' into 'master'

_loader/loader.py: Be more explanatory in CircDep exception

See merge request BuildStream/buildstream!878

f1aa0974

by Javier Jardón at 2018-10-18T14:13:35Z

Fix python warnings: Use collections.abc instead collections

Since python 3.3, collections has been moved to collections.abc module.
For backwards compatibility, they continue to be visible in this module through Python 3.7.
Subsequently, they will be removed entirely.

See https://docs.python.org/3/library/collections.html

f1550c8e

by Valentin David at 2018-10-18T16:02:45Z

Merge branch 'jjardon/warnings_collections' into 'master'

Fix python warnings: Use collections.abc instead collections

See merge request BuildStream/buildstream!883

1c24979a

by Angelos Evripiotis at 2018-10-19T21:17:01Z

Spelling fixes: 'affect', not 'effect' as verb

Fix most instances of 'effect' being used as a verb instead of a noun.
Skipped generated instance in doc/source/conf.py.

43a2eee3

by Angelos Evripiotis at 2018-10-19T21:48:59Z

Merge branch 'aevri/affect' into 'master'

Spelling fixes: 'affect', not 'effect' as verb

See merge request BuildStream/buildstream!885

12719f0d

by Jürg Billeter at 2018-10-22T17:05:41Z

_artifactcache: Rename update_atime() to update_mtime()

os.utime() updates both, however, list_artifacts() sorts refs by mtime,
i.e., atime is irrelevant.

a7a28d14

by Jürg Billeter at 2018-10-22T17:05:41Z

_artifactcache: Fix ref in update_mtime()

get_artifact_fullname() is required to construct the ref. The cache key
alone does not suffice.

be29e0f5

by Jürg Billeter at 2018-10-22T17:53:26Z

Merge branch 'juerg/cas-mtime' into 'master'

_artifactcache: Fix ref in update_mtime()

See merge request BuildStream/buildstream!882

b74aca1a

by Jürg Billeter at 2018-10-23T09:22:19Z

tests/testutils/artifactshare.py: Fix has_artifact() to match core code

Replace path separator with '-' to match Element.normal_name.

c7dda150

by Jürg Billeter at 2018-10-23T09:48:00Z

Merge branch 'juerg/test-artifactshare' into 'master'

tests/testutils/artifactshare.py: Fix has_artifact() to match core code

See merge request BuildStream/buildstream!890

74c115b9

by Angelos Evripiotis at 2018-10-23T10:07:31Z

plugins/sources: refactor, rm needless ExitStack-s

Remove two uses of ExitStack that aren't necessary and make the code
a little harder to follow.

ecb58b42

by Phil Dawson at 2018-10-23T10:33:47Z

Merge branch 'aevri/rm-exitstack' into 'master'

plugins/sources: refactor, rm needless ExitStack-s

See merge request BuildStream/buildstream!888

aa0cbf5d

by Martin Blanchard at 2018-10-23T10:54:40Z

_sandboxremote.py: Try to reopen operation steam on failure

The REAPI allows a client to reconnect to an ongoing operation stream by
providing a WaitExecution(). If implemented on server side, BuildStream
will try to recover from connection errors using it.

https://gitlab.com/BuildStream/buildstream/issues/630

552f5fc6

by Jim MacArthur at 2018-10-23T11:19:48Z

Merge branch 'mablanch/630-remote-execution-reconn' into 'master'

Handle connection losses during remote build execution

See merge request BuildStream/buildstream!806

276d2195
by Angelos Evripiotis at 2018-10-25T10:17:49Z
```
_downloadablefilesource: handle ValueError-s
```

2d012a21

by Angelos Evripiotis at 2018-10-25T10:17:49Z

Merge branch 'aevri/unknown-uri-452' into 'master'

_downloadablefilesource: handle ValueError-s

Closes #452

See merge request BuildStream/buildstream!893

cf4741ce

by James Ennis at 2018-10-25T13:05:43Z

docs: Add helpful notes to artifact server configuration instructions

5737bb06

by James Ennis at 2018-10-25T13:05:43Z

using_configuring_artifact_server.rst: Fix incorrect reference to bst push (with link)

28317622

by James Ennis at 2018-10-25T13:35:22Z

Merge branch 'jennis/patch_remote_cache_docs' into 'master'

Improve our documentation for configuring an artifact server

See merge request BuildStream/buildstream!899

d2743c52