[Notes] [Git][BuildStream/buildstream][jmac/remote_execution

Martin Blanchard pushed to branch jmac/remote_execution_client at BuildStream / buildstream

Commits:

0301f74e

by Benjamin Schubert at 2018-08-24T14:52:49Z

buildelement.py: remove double logging (again) as in #507

This was broken again by d14d8ee29b5fcaadcfd05934de71b46221243fcd,
this commits re-applies the fix from 12fea82e12a27eccb34ef4efbaa7c6874787eb4c

f991644c

by Qinusty at 2018-08-24T15:33:48Z

Merge branch 'bschubert15/fix-double-log' into 'master'

buildelement.py: remove double logging (again) as in #507

See merge request BuildStream/buildstream!730

f4c2bf19

by Phil Dawson at 2018-08-25T08:43:09Z

workspaces.rst: Move terminal sessions to doc/source/sessions-stored/

23b01f24

by Phil Dawson at 2018-08-25T08:43:09Z

workspaces.rst: fix note which is not rendering correctly

369bec0c

by Tristan Van Berkom at 2018-08-25T08:44:26Z

doc/source/developing/workspaces.rst: Some whitespace fixes

Some whitespace changes to follow the style regarding empty
lines between sections.

24cb3037

by Tristan Van Berkom at 2018-08-25T09:10:53Z

.gitignore: Ignore doc/source/sessions directory of generated html files

A preventative measure against people committing the generated
html files to the wrong location.

4f8fe977

by Tristan Van Berkom at 2018-08-25T09:10:53Z

HACKING.rst: Documenting the new "shell" configuration in the bst2html input file format

d819fb64

by Tristan Van Berkom at 2018-08-25T11:41:15Z

Merge branch 'tristan/fixup-workspace-docs' into 'master'

Fixing some fallout in the workspace documentation additions

Closes #553

See merge request BuildStream/buildstream!732

ba675863

by Tristan Van Berkom at 2018-08-25T11:42:08Z

doc/source/main_install.rst: Use one level depth toctree

This main section is just a choice of subsections, lets
make this more clear with a single level depth toctree.

52fce701

by Tristan Van Berkom at 2018-08-25T11:42:08Z

doc/source/main_install.rst: Move the main install link target here

This is referred to from the artifact install guide, and was
previously pointing directly to the source install guide.

f7289b6b

by Tristan Van Berkom at 2018-08-25T11:42:08Z

doc/source/install_source.rst: Splitting up install instructions

Instead of having all of this on one page, we now have a much
simpler page for installing on distros where BuildStream is packaged
and a separate page for installing distro specific system dependencies
and installing from source code (either from PyPI or git).

a1fa504d

by Tristan Van Berkom at 2018-08-25T11:42:08Z

Add section about current releases

This is not ideal as the releases are hardcoded, but I think is better
than nothing until we got a solution to automate this

This commit was originally by Javier Jardón <jjardon gnome org> from
merge request !661, and then later reworked into an install guide refactor.

08632568

by Tristan Van Berkom at 2018-08-25T11:42:08Z

doc/source/install_source.rst: Explain about release tag in git install instructions.

This obsoletes Laurence's explanation in merge request !657, and
is a part of the initiative of issue #528 to clarify which versions
users should be installing.

a7e046d5

by Tristan Van Berkom at 2018-08-25T12:22:51Z

Merge branch 'tristan/install-guide-refactoring' into 'master'

Install guide refactoring

See merge request BuildStream/buildstream!733

c53d190b

by Javier Jardón at 2018-08-26T06:04:58Z

doc/bst2html.py: Fix deprecation warning

This fixes the following:
"
/home/jjardon/buildstream/buildstream/doc/bst2html.py:32: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
  from collections import Mapping
"

a9390efd

by Tristan Van Berkom at 2018-08-26T07:27:50Z

Merge branch 'jjardon/doc_fix_deprecations' into 'master'

doc/bst2html.py: Fix deprecation warning

See merge request BuildStream/buildstream!727

5fb772b5

by Tristan Van Berkom at 2018-08-26T07:30:58Z

.gitlab-ci.yml: Trying to fix code quality reports

This hasnt been working for a while, it might be because GitLab
doesnt quite grasp that we have multiple branches to merge MRs to,
or it might be because they have change the API and broken back compat
in some way.

See: https://docs.gitlab.com/ee/ci/examples/code_quality.html

fa696307

by Tristan Van Berkom at 2018-08-26T08:10:59Z

Merge branch 'tristan/fix-code-quality' into 'master'

.gitlab-ci.yml: Trying to fix code quality reports

See merge request BuildStream/buildstream!735

0f329276

by Tristan Van Berkom at 2018-08-26T08:11:56Z

README.rst: Use stronger language to follow the installation guide.

And also refer to the user guide section instead of directly to the
tutorial, the tutorial will still be the first section of the user
guide so this is a better link target for getting started.

This is related to #528

4b6f3d0d

by Tristan Van Berkom at 2018-08-26T08:57:05Z

Merge branch 'tristan/readme-version' into 'master'

README.rst: Use stronger language to follow the installation guide.

See merge request BuildStream/buildstream!736

7c6e91f5

by Tristan Van Berkom at 2018-08-27T06:51:20Z

Revert ".gitlab-ci.yml: Avoid running tests in post-merge"

This reverts commit bea01531b984522940562866e904c83e8f9aed50.

539963e3

by Tristan Van Berkom at 2018-08-27T06:51:47Z

Revert "README.rst: Removing the badges"

This reverts commit da19bcf1de75bf930d0f233addc858b4073d0173.

5a5b6699

by Tristan Van Berkom at 2018-08-27T06:53:55Z

.gitlab-ci.yml: Moving code_quality job into tests phase

This job always takes a long time and does not really depend
on the artifacts in the 'test' phase, moving it to the 'test'
phase will reduce overall CI time.

c6ba8932

by Tristan Van Berkom at 2018-08-27T07:27:06Z

Merge branch 'tristan/restore-post-merge-CI' into 'master'

Tristan/restore post merge ci

See merge request BuildStream/buildstream!737

9cb69b0b

by Tristan Van Berkom at 2018-08-27T07:46:22Z

doc/source/main_install.rst: Clarify that BuildStream cannot be run on Windows

The previous wording was misleading and caused one to think you can run
BuildStream on Windows, this commit fixes that and rewords the main section
heading to be a bit more abstract.

Fixes issue #608

8c7fadd8

by Tristan Van Berkom at 2018-08-27T08:12:52Z

Merge branch 'tristan/install-guide-main' into 'master'

doc/source/main_install.rst: Clarify that BuildStream cannot be run on Windows

Closes #608

See merge request BuildStream/buildstream!739

ddbf7bc2

by Jürg Billeter at 2018-08-28T08:56:06Z

element.py: Schedule assemble for key of workspaced elements

For uncached workspaced elements, assemble is required even just to
calculate the cache key. As dynamic scheduling relies on cache keys,
schedule assemble for uncached workspace elements even if they have not
been marked as required yet.

Fixes #461.

ffc556ff

by Valentin David at 2018-08-28T09:01:05Z

tests/frontend/workspace.py: Add test for workspaced dependencies

This adds a regression test for #461.

dd90d53d

by Jürg Billeter at 2018-08-28T09:01:05Z

element.py: Do not attempt to pull workspaced elements

6a5a8e7a

by Tristan Van Berkom at 2018-08-28T11:29:22Z

Merge branch 'juerg/workspaced-dependencies' into 'master'

Fix key invalidation for workspaced dependencies

Closes #461

See merge request BuildStream/buildstream!740

6047a575

by Tristan Van Berkom at 2018-08-28T13:41:36Z

tests/frontend/workspaces.py: Removing some redundant tests

Removed redundant tests from recently merged !740, this new
test does not need to run for every different source kind.

7535fda8

by Jürg Billeter at 2018-08-28T14:25:35Z

Merge branch 'tristan/remove-redundant-test' into 'master'

tests/frontend/workspaces.py: Removing some redundant tests

See merge request BuildStream/buildstream!741

10eeb503
by Javier Jardón at 2018-08-28T22:22:00Z
```
doc/source/conf.py: Fix E201, E202 warnings
```
d3c32ca2
by Javier Jardón at 2018-08-28T22:22:00Z
```
doc/source/conf.py: Fix E402 warning
```
3ae5fd05
by Javier Jardón at 2018-08-28T22:22:00Z
```
Fix E305 warnings
```
c6fb5ba7
by Javier Jardón at 2018-08-28T22:22:00Z
```
Fix E741 warnings
```

c5eca59d

by Tristan Van Berkom at 2018-08-29T10:45:33Z

Merge branch 'jjardon/pycodestyle_fixes' into 'master'

Some pycodestyle (PEP8) fixes

See merge request BuildStream/buildstream!746

63c6ee72

by Tristan Van Berkom at 2018-08-29T10:46:30Z

doc: Adding badges.py release badge generator

This adds a step to the docs generation Makefile to generate
release.svg and snapshot.svg badges, modelled after the gitlab
badges.

This also adds the generated badges directory in docs to .gitignore

2d527052

by Tristan Van Berkom at 2018-08-29T10:46:30Z

README.rst: Adding the release and snapshot badges to the readme

This is a part of #528

29e7eea8

by Tristan Van Berkom at 2018-08-29T10:46:30Z

doc: Adding the release badges to the install page and the semantic versioning page

This is a part of #528

5d508779

by Tristan Van Berkom at 2018-08-29T11:13:53Z

Merge branch 'tristan/docs-version-badge' into 'master'

Release badges

See merge request BuildStream/buildstream!742

323a5403

by Chandan Singh at 2018-08-29T12:13:28Z

setup.py: Add more metadata useful for PyPI

- setup.py: List useful links using `project_urls`

- setup.py: Specify minumum python vresion using `python_requires`

  `python_requires` is the new way of specifying requirements for python
  version, as documented @
  https://packaging.python.org/guides/distributing-packages-using-setuptools/#python-requires.

  This will also make this requirement appear nicely on the PyPI project
  page.

- setup.py: Add project classifiers

  PyPI will use these classifiers to categorize projects while searching
  or browsing. Full list of classifiers can be found at
  https://pypi.org/classifiers.

5fc7b5ac

by Tristan Van Berkom at 2018-08-29T12:37:08Z

Merge branch 'chandan/setup.py-metadata' into 'master'

setup.py: Add more metadata useful for PyPI

See merge request BuildStream/buildstream!718

42ad937d

by Josh Smith at 2018-08-29T12:38:15Z

Prevent ValueError on URLs without an alias

Fixes #597

aa3a33b3

by Tristan Van Berkom at 2018-08-29T13:03:58Z

Merge branch 'Qinusty/597-non-alias-url-fix' into 'master'

Fix ValueError traceback for URL without alias or URI scheme

Closes #597

See merge request BuildStream/buildstream!700

2a2a79de

by Valentin David at 2018-08-29T14:11:48Z

Disable round-tripping when element is not modified

2df7607c

by Tristan Van Berkom at 2018-08-29T14:43:55Z

Merge branch 'valentindavid/roundtripping_only_when_modified' into 'master'

Disable round-tripping when element is not modified

See merge request BuildStream/buildstream!748

c146dde5
by Benjamin Schubert at 2018-08-29T16:00:24Z
```
Log not-found objects in the cache as SKIPPED
```

19e87afb

by Benjamin Schubert at 2018-08-29T16:01:12Z

Add tests that not-found objects in cache are shown as SKIPPED

9cc5817f

by Qinusty at 2018-08-29T16:25:48Z

Merge branch 'bschubert/log-missed-cache' into 'master'

Log not-found objects in the cache as SKIPPED

See merge request BuildStream/buildstream!729

3c8646e3

by Josh Smith at 2018-08-29T16:34:22Z

Add cyclic check within variable resolution

This aims to address #600, this will raise an exception when a resolved
variable contains a reference to the variable.

d41c42d6

by Josh Smith at 2018-08-29T16:34:22Z

Add tests for cyclic variables check

Note: This modifies the docker containers used for testing to supply the
pytest-timeout package.

c7b72c30

by Qinusty at 2018-08-29T16:58:48Z

Merge branch 'Qinusty/600-recursive-variables' into 'master'

Add cyclic check within variable resolution

See merge request BuildStream/buildstream!712

d28e1353

by Tristan Van Berkom at 2018-08-30T06:02:41Z

tests/plugins/filter.py: Don't run redundant tests

There is no reason that the filter element codepaths can behave
differently depending on the Source implementation used in the test,
as the Source implementation does not have any filter specific
virtual methods.

Removing the redundant tests and just performing these tests with the git source.

e77ae07a

by Tristan Van Berkom at 2018-08-30T06:36:41Z

Merge branch 'tristan/reduce-filter-tests' into 'master'

tests/plugins/filter.py: Don't run redundant tests

See merge request BuildStream/buildstream!753

9ea40e2c

by Tristan Van Berkom at 2018-08-30T06:38:13Z

tests/frontend/mirror.py: Reenable test_mirror_fetch_upstream_absent[ostree]

This test was skipped because of issue #538, but #538 was fixed
and the test was still not reenabled.

8aec1102

by Tristan Van Berkom at 2018-08-30T07:06:17Z

Merge branch 'tristan/538-reenable-ostree-test' into 'master'

tests/frontend/mirror.py: Reenable test_mirror_fetch_upstream_absent[ostree]

See merge request BuildStream/buildstream!755

fcc17c82

by Tristan Van Berkom at 2018-08-30T08:44:34Z

source.py: Stylistic changes in Source.__do_fetch()

Added some comments to make the flow easier to follow, and
removed an annoying 'success' variabled in favor of a for / else
loop statement.

60ad044a

by Tristan Van Berkom at 2018-08-30T08:44:34Z

source.py: Move Source.mark_download_url() to the public methods.

This was sitting in the section for abstract methods, but this
is most definitely not an abstract method to be implemented by
Sources.

c339f917

by Tristan Van Berkom at 2018-08-30T08:44:34Z

plugins/source/git.py: Fix formatting of url in tracking

This was displaying the aliased URL which is pretty useless,
use the translated URL for the timed activity.

26565f11

by Tristan Van Berkom at 2018-08-30T09:12:28Z

Merge branch 'tristan/source-mirroring-changes' into 'master'

Minor code changes revolving around source mirroring

See merge request BuildStream/buildstream!758

a38d9163

by Valentin David at 2018-08-30T10:26:57Z

Report processing errors from tracking

Failures to write files when tracking were not reported.

Fixes #533.

02138181

by Tristan Van Berkom at 2018-08-30T11:36:00Z

Merge branch 'valentindavid/post_tracking_errors' into 'master'

Report processing errors from tracking

Closes #533

See merge request BuildStream/buildstream!747

d23d4f3e

by Tristan Van Berkom at 2018-08-30T12:02:27Z

_artifactcache/artifactcache.py: Write the cache_size file atomically

This is causing issues while the size file is being read and written
simultaneously.

The proper fix will be to read/add/save the file atomically and that
will require locking, but this fix is a good stop gap to existing crashes.

bb6aa5a5

by Tristan Van Berkom at 2018-08-30T12:27:55Z

Merge branch 'tristan/atomic-cache-size-file' into 'master'

_artifactcache/artifactcache.py: Write the cache_size file atomically

See merge request BuildStream/buildstream!762

0a57c153