[Notes] [Git][BuildStream/buildstream][tristan/fix-status-messages] 6 co

Tristan Van Berkom pushed to branch tristan/fix-status-messages at BuildStream / buildstream

Commits:

d5b396e0

by Phillip Smyth at 2018-10-02T16:40:18Z

Forcing CI to install Sphinx 1.7.9

.gitlab-ci.yml: forced pip3 install Sphinx 1.7.9
The theme we use in our docs "sphinx_rtd_theme"
Does not support the latest version of Sphinx (>1.8)
This results in search functionality failing

dae842fd

by Phillip Smyth at 2018-10-02T17:14:09Z

Merge branch 'docs_Search_not_working' into 'master'

Fixing: Search functionality in the Documentation has stopped working

Closes #681

See merge request BuildStream/buildstream!848

8dc16d3f

by Jürg Billeter at 2018-10-03T05:08:21Z

_platform/linux.py: Accept all configs for dummy sandbox

If Sandbox.run() is never called, sandbox config is irrelevant. If it is
called, the build will fail anyway with the dummy sandbox.

This matches the logic in the Darwin platform.

66446fc3

by Jürg Billeter at 2018-10-03T05:36:38Z

Merge branch 'juerg/dummy-sandbox' into 'master'

_platform/linux.py: Accept all configs for dummy sandbox

See merge request BuildStream/buildstream!843

29c19bea

by Tristan Van Berkom at 2018-10-03T07:33:48Z

git.py: Added timed activity around fetching, displaying the URL being fetched.

This will take care of silencing the status messages while checking submodules.

b645881c

by Tristan Van Berkom at 2018-10-03T07:33:48Z

source.py: Silence messages while consuming source fetchers

The source fetchers might be a list or a generator, when it is
a generator (like the git source does), then we want to ensure that
we silence the status messages which might occur as a result of
consuming a source fetcher from the generator.

This fixes the logs to be less verbose.

Changes:

.gitlab-ci.yml

@@ -145,7 +145,8 @@ docs:
    stage: test
    script:
    - export BST_SOURCE_CACHE="$(pwd)/cache/integration-cache/sources"
 -  - pip3 install sphinx
 +  # Currently sphinx_rtd_theme does not support Sphinx >1.8, this breaks search functionality
 +  - pip3 install sphinx==1.7.9
    - pip3 install sphinx-click
    - pip3 install sphinx_rtd_theme
    - cd dist && ./unpack.sh && cd buildstream

buildstream/_platform/linux.py

@@ -55,6 +55,10 @@ class Linux(Platform):
              return SandboxBwrap(*args, **kwargs)
      def check_sandbox_config(self, config):
 +        if not self._local_sandbox_available():
 +            # Accept all sandbox configs as it's irrelevant with the dummy sandbox (no Sandbox.run).
 +            return True
++
          if self._user_ns_available:
              # User namespace support allows arbitrary build UID/GID settings.
              return True

buildstream/plugins/sources/git.py

@@ -184,10 +184,18 @@ class GitMirror(SourceFetcher):
                           cwd=self.mirror)
      def fetch(self, alias_override=None):
 -        self.ensure(alias_override)
 -        if not self.has_ref():
 -            self._fetch(alias_override)
 -        self.assert_ref()
 +        # Resolve the URL for the message
 +        resolved_url = self.source.translate_url(self.url,
 +                                                 alias_override=alias_override,
 +                                                 primary=self.primary)
++
 +        with self.source.timed_activity("Fetching from {}"
 +                                        .format(resolved_url),
 +                                        silent_nested=True):
 +            self.ensure(alias_override)
 +            if not self.has_ref():
 +                self._fetch(alias_override)
 +            self.assert_ref()
      def has_ref(self):
          if not self.ref:

buildstream/source.py

@@ -965,28 +965,48 @@ class Source(Plugin):
      # Tries to call fetch for every mirror, stopping once it succeeds
      def __do_fetch(self, **kwargs):
          project = self._get_project()
 -        source_fetchers = self.get_source_fetchers()
 +        context = self._get_context()
++
 +        # Silence the STATUS messages which might happen as a result
 +        # of checking the source fetchers.
 +        with context.silence():
 +            source_fetchers = self.get_source_fetchers()
          # Use the source fetchers if they are provided
+         #
          if source_fetchers:
 -            for fetcher in source_fetchers:
 -                alias = fetcher._get_alias()
 -                for uri in project.get_alias_uris(alias, first_pass=self.__first_pass):
 -                    try:
 -                        fetcher.fetch(uri)
 -                    # FIXME: Need to consider temporary vs. permanent failures,
 -                    #        and how this works with retries.
 -                    except BstError as e:
 -                        last_error = e
 -                        continue
+-
 -                    # No error, we're done with this fetcher
 -                    break
 -                else:
 -                    # No break occurred, raise the last detected error
 -                    raise last_error
 +            # Use a contorted loop here, this is to allow us to
 +            # silence the messages which can result from consuming
 +            # the items of source_fetchers, if it happens to be a generator.
 +            #
 +            source_fetchers = iter(source_fetchers)
 +            try:
++
 +                while True:
++
 +                    with context.silence():
 +                        fetcher = next(source_fetchers)
++
 +                    alias = fetcher._get_alias()
 +                    for uri in project.get_alias_uris(alias, first_pass=self.__first_pass):
 +                        try:
 +                            fetcher.fetch(uri)
 +                        # FIXME: Need to consider temporary vs. permanent failures,
 +                        #        and how this works with retries.
 +                        except BstError as e:
 +                            last_error = e
 +                            continue
++
 +                        # No error, we're done with this fetcher
 +                        break
++
 +                    else:
 +                        # No break occurred, raise the last detected error
 +                        raise last_error
++
 +            except StopIteration:
 +                pass
          # Default codepath is to reinstantiate the Source
+         #

[Notes] [Git][BuildStream/buildstream][tristan/fix-status-messages] 6 commits: Forcing CI to install Sphinx 1.7.9

Tristan Van Berkom pushed to branch tristan/fix-status-messages at BuildStream / buildstream

Commits:

4 changed files:

Changes: