[Notes] [Git][BuildStream/buildstream][jjardon/juerg/git-track-tags] 152

Javier Jardón pushed to branch jjardon/juerg/git-track-tags at BuildStream / buildstream

Commits:

0795a4c5

by Josh Smith at 2018-08-15T14:34:04Z

examples: Switch ftp.gnu.org to mirror

This should avoid current issues accessing the necessary sources which
are blocking CI currently.

ac97bc5e

by Valentin David at 2018-08-15T15:25:53Z

Merge branch 'Qinusty/gnu-mirror-backport-1.2' into 'bst-1.2'

Qinusty/gnu mirror backport 1.2

See merge request BuildStream/buildstream!668

d6d502f2

by Valentin David at 2018-08-16T08:37:28Z

buildstream/plugins/sources/git.py: Fix clone side effect in getting fetchers

We now delay refreshing submodules until we have a clone, that is after
we have fetched the main fetcher.

Fixes #537

31ee84ac

by Valentin David at 2018-08-16T08:37:28Z

tests/frontend/mirror.py: Re-enable a test for git

debdd7dd

by Jonathan Maw at 2018-08-16T08:37:28Z

tests: Test that fetching passes when upstream is absent

3d9aac51

by Jonathan Maw at 2018-08-16T08:37:28Z

tests: Add a test of the git source with submodules

de5ca03d

by Valentin David at 2018-08-16T08:37:28Z

Test git mirroring fallback on submodules when main repo is not mirrored.

6023004b

by Valentin David at 2018-08-16T08:37:28Z

Test we can discovered submodules on fallback mirrored git repositories

b77305d3

by Valentin David at 2018-08-16T11:14:52Z

Merge branch 'valentindavid/fallback_mirror_git-1.2' into 'bst-1.2'

Delay refreshing git submodule until we have a clone

See merge request BuildStream/buildstream!666

449a692b

by Tiago Gomes at 2018-08-16T12:03:52Z

Use http instead of https for the gnu ftp mirror

https seems broken on the Debian image:

START   autotools/hello/42930621-fetch.499.log
START   Fetching https://ftpmirror.gnu.org/gnu/automake/automake-1.16.tar.gz
FAILURE Fetching https://ftpmirror.gnu.org/gnu/automake/automake-1.16.tar.gz
FAILURE tar source at hello.bst [line 16 column 2]: Error mirroring https://ftpmirror.gnu.org/gnu/automake/automake-1.16.tar.gz: <urlopen error [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:720)>

3bc8a513

by Tiago Gomes at 2018-08-16T12:58:32Z

Merge branch 'tiagogomes/disable-https-gnu-repo-backport' into 'bst-1.2'

Use http instead of https for the gnu ftp mirror

See merge request BuildStream/buildstream!677

de73cc0b

by Valentin David at 2018-08-16T13:37:06Z

Fix ostree repository mirroring

Ostree mirrors were not sharing the same local repository, so it was
impossible the request refs from the right local repository when data
was fetched from a mirror rather than upstream.

Instead of having several repository with one remote each, we now
have one repository with several remotes.

This fixes #538.

43f4ffba

by Valentin David at 2018-08-16T14:36:49Z

Merge branch 'valentindavid/fallback_mirror_ostree-1.2' into 'bst-1.2'

Fix ostree repository mirroring

See merge request BuildStream/buildstream!667

1e140f7d

by Tristan Van Berkom at 2018-08-16T14:38:02Z

_frontend/app.py: Notify session completions

Use the optionally implemented desktop notification method
to notify when a session completes.

Previously, this used to only notify when an interactive prompt
appears.

This is an attempt to fix #385

87960822

by Tristan Van Berkom at 2018-08-16T15:36:18Z

Merge branch 'tristan/notifications-1.2' into 'bst-1.2'

_frontend/app.py: Notify session completions

See merge request BuildStream/buildstream!673

3200093e

by Tiago Gomes at 2018-08-16T16:20:57Z

cascache: use errno module

os.errno does no longer work with Python 3.7

Closes #577.

6c35850b

by Javier Jardón at 2018-08-16T17:11:29Z

Merge branch 'tiagogomes/issue-577-backport' into 'bst-1.2'

cascache: use errno module

See merge request BuildStream/buildstream!675

27220c69

by William Salmon at 2018-08-17T09:28:44Z

Add Error to git and ostree sources configure

Raise a error at configure time if the track and ref properties are
not present in the sources.

This is to address https://gitlab.com/BuildStream/buildstream/issues/471
that documented unhelpful behaviour when tracking git sources. However
the issue was also identified in ostree.

22b6fa21

by William Salmon at 2018-08-17T09:28:44Z

Improve error message for build if there are refs missing

eda495a5

by Will Salmon at 2018-08-17T10:28:27Z

Merge branch 'willsalmon/580-backport' into 'bst-1.2'

Add warning to git track if track and ref are not present

See merge request BuildStream/buildstream!621

ef5f6a61

by Josh Smith at 2018-08-20T08:12:14Z

job.py: Prevent terminated jobs retrying

Fixes #531: Jobs were retrying when terminated, this lead to the process
being spawned again and starting up fresh.

ef103572

by Qinusty at 2018-08-20T10:06:48Z

Merge branch 'Qinusty/terminate-retries-backport-1.2' into 'bst-1.2'

Backport: Prevent terminated jobs retrying

See merge request BuildStream/buildstream!681

83f4403d

by Valentin David at 2018-08-20T15:39:06Z

Fix broken indentation after tracking.

Issue was introduced by 171e803f (include directive) and the fix was
found courtesy of @Qinusty. This fixes also the include
feature. Because elements are to be serialized, the included fragments
need to use copy_tree when loaded.

Related to #470.

d25b7d77

by Valentin David at 2018-08-20T15:39:07Z

buildstream/_includes.py: Fix caching of included fragments.

ce6d5f8c

by Valentin David at 2018-08-20T17:02:58Z

Merge branch 'valentindavid/fix-broken-indentation-after-track-1.2' into 'bst-1.2'

Fix broken indentation after track

See merge request BuildStream/buildstream!687

24da69cf

by Chandan Singh at 2018-08-21T12:30:07Z

.gitlab-ci.yml: Bump tags for testsuite images

These new images come with pinned versions of `pytest` and `pylint`:

- `pylint`: `== 2.1.1`
- `pytest`: `>= 3.7`

54d15225

by Chandan Singh at 2018-08-21T12:30:13Z

dev-requirements.txt: Pin versions of pytest and pylint

These packages should only be updated as a deliberate change and the
code should be tested with the new versions to ensure that the test
still pass.

Any future updates should also happen in sync with the
buildstream-docker-images repository so that the testsuite base images
also enforce the same restrictions for the development requirements.

Replaces https://gitlab.com/BuildStream/buildstream/merge_requests/684.

be7533ab

by Chandan Singh at 2018-08-21T13:31:52Z

Merge branch 'chandan/update-ci-tags-1.2' into 'bst-1.2'

Restrict pylint/pytest versions and bump CI tags

See merge request BuildStream/buildstream!690

1be685a7

by Tristan Van Berkom at 2018-08-21T13:43:27Z

_frontend/linuxapp.py: Fix special casing around desktop notification escape sequence

Now we allow the notification to happen on any TERM
which starts with 'xterm' or 'vte', and we only do it
if the VTE_VERSION is >= 4600, where we know for sure that
VTE will not print garbage on the terminal.

Fixes #385

fc3cff95

by Tristan Van Berkom at 2018-08-21T15:44:21Z

Merge branch 'tristan/notifications-1.2' into 'bst-1.2'

_frontend/linuxapp.py: Fix special casing around desktop notification escape sequence

See merge request BuildStream/buildstream!693

6f6a04a9

by Jonathan Maw at 2018-08-22T05:54:45Z

Pipeline: Skip except_elements logic if no elements to except

1d48dee9

by Tristan Van Berkom at 2018-08-22T06:51:51Z

Merge branch 'jonathan/faster-except-elements-1.2' into 'bst-1.2'

Pipeline: Skip except_elements logic if no elements to except

See merge request BuildStream/buildstream!691

a4a8e5ab

by William Salmon at 2018-08-22T08:07:03Z

Catch Non Numeric versions

This patch just displays a better message than the default stack trace
but dose not try to fix the problem. A further patch will be created but
it effects versioneer so may take longer to land as it may need to go
via versioneer mainline.

3747f5d1

by William Salmon at 2018-08-22T08:07:03Z

Search for tags with the *.*.* patten for version

This has been done in a way that I hope might be able to be upstreamed
in to versioneer. This is not garanteed but it would be good if it
could be.

Please see https://github.com/warner/python-versioneer/pull/186 and
https://github.com/wsalmonct/python-versioneer/tree/feature/regexTags
for further details.

794b73e3

by Will Salmon at 2018-08-22T09:31:24Z

Merge branch 'willsalmon/MR601-backport' into 'bst-1.2'

Willsalmon/mr601 backport

See merge request BuildStream/buildstream!696

65cdd6dc

by Chandan Singh at 2018-08-22T10:08:52Z

setup.py: Add fields required for publishing on PyPi

Fill out those fields in `setup.py` that are required for publishing a
package on PyPi.

Part of https://gitlab.com/BuildStream/buildstream/issues/587.

2872bc6d

by Chandan Singh at 2018-08-22T11:11:36Z

Merge branch 'chandan/pypi-prep-1.2' into 'bst-1.2'

setup.py: Add fields required for publishing on PyPi

See merge request BuildStream/buildstream!699

0bcfdfb9

by Sam Thursfield at 2018-08-22T11:36:29Z

Don't strip pathname from debug symbol files

When GDB looks for debug symbols it looks for the full path of the file
inside the configured debug-file-directory. For example, if the
debug-file-directory is set to a default of /usr/lib/debug, and you are
debugging /usr/bin/python3, GDB will look for its debug symbols at this
location: /usr/lib/debug/usr/bin/python3.

BuildStream has been putting all debug files inside /usr/lib/debug
under their $(basename), so in the above example GDB would fail to find
any debug symbols for /usr/bin/python3 because they would be in the
incorrect locatoin of /usr/lib/debug/python3.

b567666d

by Tristan Van Berkom at 2018-08-22T13:04:22Z

Merge branch 'tristan/debug-symbols-location-1.2' into 'bst-1.2'

Don't strip pathname from debug symbol files

See merge request BuildStream/buildstream!698

6a7fdf12

by Jonathan Maw at 2018-08-22T13:22:27Z

Use ArtifactCache's get_cache_size when calculating the quota

It would have been more convenient to store the quota size in the
ArtifactCache, except that the scheduler needs the quota size and
doesn't have access to the ArtifactCache.

baee80b4

by Valentin David at 2018-08-22T17:28:55Z

Merge branch 'valentindavid/faster_startup-1.2' into 'bst-1.2'

Use ArtifactCache's get_cache_size when calculating the quota

See merge request BuildStream/buildstream!701

e1d5bfa5

by Chandan Singh at 2018-08-22T20:11:19Z

MANIFEST.in: Fix include for dev-requirements.txt

Currently, running any setuptools commands using `setup.py` generates a
warning like so:
        warning: manifest_maker: MANIFEST.in, line 21: unknown action 'dev-requirements.txt'

The syntax is invalid and got accidentally introduced in !637 but it's a
shame that this is not a warning and no obvious way to make it an error
if there are invalid things in MANIFEST.in file.

---

This is a backport of
https://gitlab.com/BuildStream/buildstream/merge_requests/704.

be8cb8ae

by Chandan Singh at 2018-08-22T23:44:28Z

Merge branch 'chandan/fix-mainfest-dev-requirements-1.2' into 'bst-1.2'

MANIFEST.in: Fix include for dev-requirements.txt

See merge request BuildStream/buildstream!705

a53b7e21

by Tristan Van Berkom at 2018-08-23T05:51:15Z

_frontend/linuxapp.py: Fixing fallout from !693

When fixing terminal notifications, I had introduced a bug
with accesses to `os.environ` which triggered KeyError, this
patch fixes it.

f8cfcb7c

by Jonathan Maw at 2018-08-23T05:55:06Z

Fix get_cache_size to store it in the ArtifactCache, not the context

dd4f7ced

by Jonathan Maw at 2018-08-23T05:55:06Z

artifactcache: Read and write cache size to disk

e3cfcdea
by Jonathan Maw at 2018-08-23T05:55:06Z
```
Add tests that cache size works
```

1a344adc

by Tristan Van Berkom at 2018-08-23T06:56:55Z

Merge branch 'tristan/finish-backport-of-679' into 'bst-1.2'

Tristan/finish backport of 679

See merge request BuildStream/buildstream!706

2a33f557

by Valentin David at 2018-08-23T07:06:20Z

Improve error message for deleted open workspaces

Fixes #576.

09edbeea

by Tristan Van Berkom at 2018-08-23T08:00:28Z

Merge branch 'tristan/inconsistent-workspace-1.2' into 'bst-1.2'

Improve error message for deleted open workspaces

See merge request BuildStream/buildstream!707

3e1ea8d9

by Valentin David at 2018-08-23T08:04:27Z

Remove artifact extracts when artifact expires in cache

Fixes #561

7c84d089

by Tristan Van Berkom at 2018-08-23T09:03:18Z

Merge branch 'tristan/artifact-expiry-1.2' into 'bst-1.2'

Remove artifact extracts when artifact expires in cache

See merge request BuildStream/buildstream!708

a5aa18a4
by Tristan Van Berkom at 2018-08-23T09:16:50Z
```
.gitlab-ci.yml: Avoid running tests in post-merge
```

e1648312

by Tristan Van Berkom at 2018-08-23T09:17:00Z

README.rst: Removing the badges

Now that we only run pre-merge CI, it makes no sense to show a badge for
the coverage and pipeline status of the last run of master.

It would make sense to show a badge for the last pipeline which
was run for the last merge request that was merged to master, but
I don't see any feature to get that information.

d02e36b8

by Tristan Van Berkom at 2018-08-23T11:04:32Z

Merge branch 'tristan/reduce-gitlab-ci-1.2' into 'bst-1.2'

.gitlab-ci.yml: Avoid running tests in post-merge (1.2)

See merge request BuildStream/buildstream!710

add9645c

by Jürg Billeter at 2018-08-23T11:05:04Z

_artifactcache/casserver.py: Fix resource_name format for blobs

Continue to accept requests from non-conforming BuildStream clients for
the time being to ease migration.

96ff4c06

by Jürg Billeter at 2018-08-23T11:05:04Z

_artifactcache/cascache.py: Fix resource_name format for blobs

This requires an updated server.

Fixes #572.

0eaa5257

by Jürg Billeter at 2018-08-23T11:05:31Z

NEWS: Add news entry for CAS resource_name format change

037b8dd4

by Jürg Billeter at 2018-08-23T11:05:31Z

_artifactcache/casserver.py: Improve ByteStream error handling

Replace assertions with gRPC error responses.

d9a97c33

by Jonathan Maw at 2018-08-23T11:05:31Z

CASCache: Fix cell-var-from-loop linting error

9983fc3a

by Jürg Billeter at 2018-08-23T11:58:25Z

Merge branch 'juerg/cas-1.2' into 'bst-1.2'

CAS: Fix resource_name format for blobs

See merge request BuildStream/buildstream!711

301a393c
by Tristan Van Berkom at 2018-08-23T12:38:10Z
```
NEWS: Updating for 1.1.7 release
```

d824489b

by Valentin David at 2018-08-24T07:40:56Z

tests/integration/source-determinism.py: Use cli_integration.

e04178da

by Tristan Van Berkom at 2018-08-24T08:21:50Z

Merge branch 'tristan/integration-cli-source-determinism' into 'bst-1.2'

tests/integration/source-determinism.py: Use cli_integration.

See merge request BuildStream/buildstream!721

da901c86

by Tristan Van Berkom at 2018-08-24T08:54:27Z

tests/integration/project: Use the same alias for the alpine tarball

This makes the integration tests use the same 'alpine' alias for the
tests as we use in the examples, this avoids a redundant download
of an extra alpine tarball in integration tests.

This is a part of #603

a201f9a9

by Tristan Van Berkom at 2018-08-24T09:30:53Z

Merge branch 'tristan/avoid-redundant-alpine-1.2' into 'bst-1.2'

tests/integration/project: Use the same alias for the alpine tarball

See merge request BuildStream/buildstream!723

72587820

by Josh Smith at 2018-08-24T12:42:40Z

setup.cfg: Show timing of the 20 slowest tests

This helps identifying what tests are slow or if the CI got stuck on
some of them

bd96e353

by Javier Jardón at 2018-08-24T13:08:02Z

Merge branch 'jjardon/ci_show_timings-bst-1.2' into 'bst-1.2'

[bst-1.2] .gitlab-ci.yml: Show timing of the 20 slowest tests

See merge request BuildStream/buildstream!728

c3ebd59a

by Javier Jardón at 2018-08-26T06:07:32Z

doc/bst2html.py: Fix deprecation warning

This fixes the following:
"
/home/jjardon/buildstream/buildstream/doc/bst2html.py:32: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
  from collections import Mapping
"

81261e3c

by Tristan Van Berkom at 2018-08-26T07:19:33Z

Merge branch 'tristan/doc-fix-deprecations-1.2' into 'bst-1.2'

doc/bst2html.py: Fix deprecation warning (1.2)

See merge request BuildStream/buildstream!734

c54e5ff7

by Tristan Van Berkom at 2018-08-27T07:00:09Z

.gitlab-ci.yml: Trying to fix code quality reports

This hasnt been working for a while, it might be because GitLab
doesnt quite grasp that we have multiple branches to merge MRs to,
or it might be because they have change the API and broken back compat
in some way.

See: https://docs.gitlab.com/ee/ci/examples/code_quality.html

0fd3d96e

by Tristan Van Berkom at 2018-08-27T07:00:26Z

Revert ".gitlab-ci.yml: Avoid running tests in post-merge"

This reverts commit bea01531b984522940562866e904c83e8f9aed50.

97373d2f

by Tristan Van Berkom at 2018-08-27T07:00:33Z

Revert "README.rst: Removing the badges"

This reverts commit da19bcf1de75bf930d0f233addc858b4073d0173.

2f36d185

by Tristan Van Berkom at 2018-08-27T07:00:40Z

.gitlab-ci.yml: Moving code_quality job into tests phase

This job always takes a long time and does not really depend
on the artifacts in the 'test' phase, moving it to the 'test'
phase will reduce overall CI time.

f3ebfe35

by Tristan Van Berkom at 2018-08-27T07:31:27Z

Merge branch 'tristan/restore-post-merge-CI-1.2' into 'bst-1.2'

Restore post merge CI in 1.2

See merge request BuildStream/buildstream!738

76a66abf

by Jürg Billeter at 2018-08-28T13:29:47Z

element.py: Schedule assemble for key of workspaced elements

For uncached workspaced elements, assemble is required even just to
calculate the cache key. As dynamic scheduling relies on cache keys,
schedule assemble for uncached workspace elements even if they have not
been marked as required yet.

Fixes #461.

1a96b537

by Valentin David at 2018-08-28T13:29:48Z

tests/frontend/workspace.py: Add test for workspaced dependencies

This adds a regression test for #461.

765dd3d9

by Jürg Billeter at 2018-08-28T13:29:48Z

element.py: Do not attempt to pull workspaced elements

0de2f87b

by Tristan Van Berkom at 2018-08-28T13:39:18Z

tests/frontend/workspaces.py: Removing some redundant tests

Removed redundant tests from recently merged !740, this new
test does not need to run for every different source kind.

7da1c309

by Jürg Billeter at 2018-08-28T16:32:34Z

Merge branch 'juerg/workspaced-dependencies-1.2' into 'bst-1.2'

Fix key invalidation for workspaced dependencies

See merge request BuildStream/buildstream!744

291763b0

by Chandan Singh at 2018-08-29T12:23:21Z

setup.py: Add more metadata useful for PyPI

- setup.py: List useful links using `project_urls`

- setup.py: Specify minumum python vresion using `python_requires`

  `python_requires` is the new way of specifying requirements for python
  version, as documented @
  https://packaging.python.org/guides/distributing-packages-using-setuptools/#python-requires.

  This will also make this requirement appear nicely on the PyPI project
  page.

- setup.py: Add project classifiers

  PyPI will use these classifiers to categorize projects while searching
  or browsing. Full list of classifiers can be found at
  https://pypi.org/classifiers.

557fb81a

by Tristan Van Berkom at 2018-08-29T12:44:25Z

Merge branch 'tristan/setup.py-metadata-1.2' into 'bst-1.2'

setup.py: Add more metadata useful for PyPI

See merge request BuildStream/buildstream!749

582e0c05

by Josh Smith at 2018-08-29T13:02:04Z

Prevent ValueError on URLs without an alias

Fixes #597

4dfb4169

by Qinusty at 2018-08-29T14:08:58Z

Merge branch 'Qinusty/597-backport-ValueError-fix' into 'bst-1.2'

Backport !700:  Prevent ValueError on URLs without an alias

See merge request BuildStream/buildstream!751

c09852c9
by Benjamin Schubert at 2018-08-29T16:58:43Z
```
Log not-found objects in the cache as SKIPPED
```

cba9208f

by Benjamin Schubert at 2018-08-29T16:58:51Z

Add tests that not-found objects in cache are shown as SKIPPED

4a644deb

by Tristan Van Berkom at 2018-08-29T17:24:18Z

Merge branch 'bschubert/backport-log-missed-cache' into 'bst-1.2'

Bschubert/backport log missed cache

See merge request BuildStream/buildstream!752

8498de5a

by Tristan Van Berkom at 2018-08-30T06:05:28Z

tests/plugins/filter.py: Don't run redundant tests

There is no reason that the filter element codepaths can behave
differently depending on the Source implementation used in the test,
as the Source implementation does not have any filter specific
virtual methods.

Removing the redundant tests and just performing these tests with the git source.

9aeed43c

by Tristan Van Berkom at 2018-08-30T06:34:00Z

Merge branch 'tristan/reduce-filter-tests-1.2' into 'bst-1.2'

tests/plugins/filter.py: Don't run redundant tests

See merge request BuildStream/buildstream!754

cd608e28

by Tristan Van Berkom at 2018-08-30T06:39:54Z

tests/frontend/mirror.py: Reenable test_mirror_fetch_upstream_absent[ostree]

This test was skipped because of issue #538, but #538 was fixed
and the test was still not reenabled.

355f9208

by Tristan Van Berkom at 2018-08-30T07:07:48Z

Merge branch 'tristan/538-reenable-ostree-test-1.2' into 'bst-1.2'

tests/frontend/mirror.py: Reenable test_mirror_fetch_upstream_absent[ostree]

See merge request BuildStream/buildstream!756

d67d14f1

by Josh Smith at 2018-08-30T08:03:08Z

Add cyclic check within variable resolution

This aims to address #600, this will raise an exception when a resolved
variable contains a reference to the variable.

c8b422f1

by Josh Smith at 2018-08-30T08:03:17Z

Add tests for cyclic variables check

Note: This modifies the docker containers used for testing to supply the
pytest-timeout package.

0e26667e

by Qinusty at 2018-08-30T08:33:09Z

Merge branch 'Qinusty/cyclic-variable-backport' into 'bst-1.2'

Backport cyclic variable fix

See merge request BuildStream/buildstream!757

68b44195

by Tristan Van Berkom at 2018-08-30T08:51:20Z

source.py: Stylistic changes in Source.__do_fetch()

Added some comments to make the flow easier to follow, and
removed an annoying 'success' variabled in favor of a for / else
loop statement.

88bdf22a

by Tristan Van Berkom at 2018-08-30T08:51:41Z

source.py: Move Source.mark_download_url() to the public methods.

This was sitting in the section for abstract methods, but this
is most definitely not an abstract method to be implemented by
Sources.

813fcd60

by Tristan Van Berkom at 2018-08-30T08:52:43Z

plugins/source/git.py: Fix formatting of url in tracking

This was displaying the aliased URL which is pretty useless,
use the translated URL for the timed activity.

3086eda5

by Tristan Van Berkom at 2018-08-30T09:17:43Z

Merge branch 'tristan/source-mirroring-changes-1.2' into 'bst-1.2'

Minor code changes revolving around source mirroring

See merge request BuildStream/buildstream!759

57c5f837

by Valentin David at 2018-08-30T10:43:24Z

Disable round-tripping when element is not modified

017821c7

by Tristan Van Berkom at 2018-08-30T11:03:17Z

Merge branch 'valentindavid/roundtripping_only_when_modified-1.2' into 'bst-1.2'

Disable round-tripping when element is not modified

See merge request BuildStream/buildstream!760

183aa62a

by Valentin David at 2018-08-30T11:38:38Z

Report processing errors from tracking

Failures to write files when tracking were not reported.

Fixes #533.

82305517

by Valentin David at 2018-08-30T12:10:22Z

Merge branch 'valentindavid/post_tracking_error-1.2' into 'bst-1.2'

Report processing errors from tracking

See merge request BuildStream/buildstream!761

6d438496

by Tristan Van Berkom at 2018-08-30T12:10:40Z

_artifactcache/artifactcache.py: Write the cache_size file atomically

This is causing issues while the size file is being read and written
simultaneously.

The proper fix will be to read/add/save the file atomically and that
will require locking, but this fix is a good stop gap to existing crashes.

b83f7fa1

by Tristan Van Berkom at 2018-08-30T12:32:22Z

Merge branch 'tristan/atomic-cache-size-file-1.2' into 'bst-1.2'

_artifactcache/artifactcache.py: Write the cache_size file atomically

See merge request BuildStream/buildstream!763

127893fa

by Tom Pollard at 2018-08-31T06:26:44Z

buildstream/_project.py: Report if project.conf is missing name

Explicitly check that project.conf contains a name. This resolves
the issue of the provenance check from _yaml.py incorrectly reporting
the offending file as the default_config_node projectconfig.yaml
when no name key exists in the pre_config_node dict.

f861dc0b

by Tristan Van Berkom at 2018-08-31T06:26:52Z

_project.py: Adding FIXME comment to address #591 properly

2b94532a

by Tristan Van Berkom at 2018-08-31T06:51:54Z

Merge branch 'tristan/591-workaround-1.2' into 'bst-1.2'

Backport workaround for #591

See merge request BuildStream/buildstream!767

0dc75244

by Josh Smith at 2018-08-31T11:29:38Z

job.py: Modify retry messages to be FAIL

This adjusts the message handler for the child processes to no longer
override the message type.

This also removes the ability for unhandled non BstError exceptions to
retry.

d013bb2c

by Josh Smith at 2018-08-31T11:29:46Z

tests: Modify tests to ensure retry FAILURE

68275614

by Qinusty at 2018-08-31T11:54:08Z

Merge branch 'Qinusty/retries-should-fail-backport' into 'bst-1.2'

Retries should fail Backport 1.2

See merge request BuildStream/buildstream!768

9e86c8d2

by Josh Smith at 2018-08-31T15:57:35Z

scheduler.py: Move cache_size logs into folder

This prevents the cache_size.pid.log files from cluttering the root
log directory.

b97a92b0

by Tristan Van Berkom at 2018-09-01T08:24:54Z

Merge branch 'Qinusty/cache-size-directory-backport' into 'bst-1.2'

Backport: Move cache_size.pid.log files into a subdirectory of logs

See merge request BuildStream/buildstream!770

2a27b9c4

by Tristan Van Berkom at 2018-09-03T06:18:17Z

source.py: Document Source.get_source_fetchers() to return an iterable

Also highlight the fact that the plugin can rely on the fetcher's
fetch() method getting called before consuming the next item in the
list, which is the magick behavior that the git plugin relies on.

This is a part of #620

a6dbf5fc

by Tristan Van Berkom at 2018-09-03T06:18:23Z

source.py: Fixing docs link formatting to be consistent.

62b7ec45

by Tristan Van Berkom at 2018-09-03T06:18:31Z

source.py: Documenting that marking download URLs is mandatory

A download URL must be interpreted by the core at `Plugin.configure()`
time, even if only employed later on.

This is a part of #620

4df625c8

by Tristan Van Berkom at 2018-09-03T06:19:48Z

plugin.py: Added _configure() and _get_configuring() private APIs

Keeps track of whether the plugin is currently being configured.

Adjusted Element and Source classes to call _configure() in place
of calling configure() directly.

This is a part of #620

435dc56c

by Tristan Van Berkom at 2018-09-03T06:20:13Z

source.py: Added `primary` argument to URL marking APIs

The Source must now mention whether the marked or translated
URL is "primary" or not. Even when a Source may have multiple
URLs, the auxilliary URLs are derived from the primary one, not
only is this true for git, but it is mandated by our tracking
API which assumes there is a primary URL.

This adjusts the `git` source and the test `fetch_source.py` source
to behave properly and advertize it's primary URL properly.

This is a part of #620

51eaa2c3

by Tristan Van Berkom at 2018-09-03T06:20:27Z

source.py: Track marked URLs and assert they are marked during Plugin.configure()

This cannot test for unaliased URLs, as those can be discovered later
on outside of user provided element configuration; at least we
assert that if an alias was used, we have seen it at load time.

This will cause a BUG to occur for a plugin which marks an aliased
URL (or attempts to translate one) outside of `Plugin.configure()`,
if that URL was not previously seen.

This is a part of #620

b422959f

by Tristan Van Berkom at 2018-09-03T06:50:11Z

Merge branch 'tristan/source-fetcher-changes-1.2' into 'bst-1.2'

Source fetcher changes 1.2

See merge request BuildStream/buildstream!773

25c6ef05

by Tristan Van Berkom at 2018-09-03T07:04:25Z

doc/examples/first-project/project.conf: Updating format version

This is a workaround for a bug that this regenerated file causes
changes to the git repository while building documentation

3f24587f

by Tristan Van Berkom at 2018-09-03T07:04:25Z

docs: Updating static session files for the 1.2.0 release

We regenerate these with releases so that packagers dont
need to rebuild them, because rebuilding these sessions requires
external resources and we expect packagers to build the package
in restricted environments without internet access.

287c8f1d

by Tristan Van Berkom at 2018-09-03T07:25:15Z

Merge branch 'tristan/update-docs' into 'bst-1.2'

Updating docs session files before the release

See merge request BuildStream/buildstream!774

70971f24
by Tristan Van Berkom at 2018-09-03T07:45:24Z
```
NEWS: Updating for the 1.2.0 release
```

80e912e6

by Tristan Van Berkom at 2018-09-10T06:56:11Z

tests/artifactcache/expiry.py: Fix test case expectations.

f81637c5

by Tristan Van Berkom at 2018-09-10T06:56:11Z

_scheduler/queues: Mark build and pull queue as requiring shared access to the CACHE

This is what the whole resource.py thing was created for, the
cleanup job must have exclusive access to the cache, while the pull
and build jobs which result in adding new artifacts, must only require
shared access.

This is a part of #623

f074a8ac

by Tristan Van Berkom at 2018-09-10T06:56:11Z

_scheduler/resources.py: Set CACHE max resources to 0

This means there is no cap for shared resource requests.

Together with the previous commit, this causes the cleanup
job and the pull/build jobs to all require unlimited shared
access to the CACHE resource; while only the cleanup job
requires exclusive access to the resource.

This is a part of #623

d2f5c4f3