[Notes] [Git][BuildStream/buildstream][tlater/message-lines] 6 commits:

Tristan Van Berkom pushed to branch tlater/message-lines at BuildStream / buildstream

Commits:

adfb9291

by Tristan Van Berkom at 2019-01-18T21:23:05Z

tests/testutils/python_repo.py: Use subprocess to run sdist

The current approach using setuptools.sandbox.run_setup() was
causing a spurious (but highly frequent) failure where setuptools
gets mixed up with it's manipulation of sys.modules and hits a
RuntimeError as a dictionary changes size while being iterated over.

For instance: https://gitlab.com/BuildStream/buildstream/-/jobs/147967307

Since this already happens in an isolated virtual environment created
by tox, we should not need additional sandboxing here from setuptools,
and launching this as a subprocess will be safer.

d114a6bd

by Tristan Van Berkom at 2019-01-18T22:07:22Z

Merge branch 'tristan/fix-pip-source-test' into 'master'

tests/testutils/python_repo.py: Use subprocess to run sdist

See merge request BuildStream/buildstream!1090

bf591ade

by Chandan Singh at 2019-01-18T22:09:04Z

Derive import plugin from Element instead of BuildElement

The `import` element is not really a build element. The main purpose of
the `BuildElement` class is to allow users to run `build-commands`,
`install-commands` etc. But, `import` does not run such commands.
Moreover, we already override all the methods provided by
`BuildElement`. So it only makes it confusing to have it derived from
`BuildElement` class when it is not a build element.

So, derive it from the base `Element` class instead.

2233a532

by Chandan Singh at 2019-01-18T22:46:41Z

Merge branch 'chandan/import-is-not-buildelement' into 'master'

Derive import plugin from Element instead of BuildElement

See merge request BuildStream/buildstream!1089

aa3411f9

by Tristan Maat at 2019-01-18T22:47:46Z

testutils/runcli.py: Allow removing artifacts from arbitrary dirs

`remove_artifact_from_cache` used a hard-coded path to remove
artifacts, which wasn't sufficient for integration tests.

fff882fe

by Tristan Maat at 2019-01-18T22:47:46Z

widget.py: Avoid "showing 0 lines" messages when there are no lines

This happened when bst is invoked with --message-lines 0 or
--error-lines 0, and was arguably a little too verbose (the user
explicitly asked us not to show them any lines, after all).

Fixes #779

Changes:

buildstream/_frontend/widget.py

@@ -647,8 +647,9 @@ class LogLine(Widget):
              abbrev = False
              if message.message_type not in ERROR_MESSAGES \
                 and not frontend_message and n_lines > self._message_lines:
 -                abbrev = True
                  lines = lines[0:self._message_lines]
 +                if self._message_lines > 0:
 +                    abbrev = True
              else:
                  lines[n_lines - 1] = lines[n_lines - 1].rstrip('\n')
@@ -674,7 +675,7 @@ class LogLine(Widget):
              if self.context is not None and not self.context.log_verbose:
                  text += self._indent + self._err_profile.fmt("Log file: ")
                  text += self._indent + self._logfile_widget.render(message) + '\n'
 -            else:
 +            elif self._log_lines > 0:
                  text += self._indent + self._err_profile.fmt("Printing the last {} lines from log file:"
                                                               .format(self._log_lines)) + '\n'
                  text += self._indent + self._logfile_widget.render(message, abbrev=False) + '\n'

buildstream/plugins/elements/import.py

@@ -28,17 +28,14 @@ some configuration data.
  The empty configuration is as such:
    .. literalinclude:: ../../../buildstream/plugins/elements/import.yaml
       :language: yaml
+-
 -See :ref:`built-in functionality documentation <core_buildelement_builtins>` for
 -details on common configuration options for build elements.
  """
  import os
 -from buildstream import Element, BuildElement, ElementError
 +from buildstream import Element, ElementError
  # Element implementation for the 'import' kind.
 -class ImportElement(BuildElement):
 +class ImportElement(Element):
      # pylint: disable=attribute-defined-outside-init
      # This plugin has been modified to avoid the use of Sandbox.get_directory
@@ -93,10 +90,6 @@ class ImportElement(BuildElement):
          # And we're done
          return '/output'
 -    def prepare(self, sandbox):
 -        # We inherit a non-default prepare from BuildElement.
 -        Element.prepare(self, sandbox)
+-
      def generate_script(self):
          build_root = self.get_variable('build-root')
          install_root = self.get_variable('install-root')

tests/integration/messages.py

 +#
 +#  Copyright (C) 2018 Codethink Limited
 +#
 +#  This program is free software; you can redistribute it and/or
 +#  modify it under the terms of the GNU Lesser General Public
 +#  License as published by the Free Software Foundation; either
 +#  version 2 of the License, or (at your option) any later version.
 +#
 +#  This library is distributed in the hope that it will be useful,
 +#  but WITHOUT ANY WARRANTY; without even the implied warranty of
 +#  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
 +#  Lesser General Public License for more details.
 +#
 +#  You should have received a copy of the GNU Lesser General Public
 +#  License along with this library. If not, see <http://www.gnu.org/licenses/>.
 +#
 +#  Authors: Tristan Maat <tristan maat codethink co uk>
 +#
++
 +import os
 +import pytest
++
 +from buildstream import _yaml
 +from buildstream._exceptions import ErrorDomain
++
 +from tests.testutils import cli_integration as cli
 +from tests.testutils.site import HAVE_BWRAP, IS_LINUX
++
++
 +pytestmark = pytest.mark.integration
++
++
 +# Project directory
 +DATA_DIR = os.path.join(
 +    os.path.dirname(os.path.realpath(__file__)),
 +    "project",
 +)
++
++
 +@pytest.mark.integration
 +@pytest.mark.datafiles(DATA_DIR)
 +@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
 +def test_disable_message_lines(cli, tmpdir, datafiles):
 +    project = os.path.join(datafiles.dirname, datafiles.basename)
 +    element_path = os.path.join(project, 'elements')
 +    element_name = 'message.bst'
++
 +    element = {
 +        'kind': 'manual',
 +        'depends': [{
 +            'filename': 'base.bst'
 +        }],
 +        'config': {
 +            'build-commands':
 +            ['echo "Silly message"'],
 +            'strip-commands': []
 +        }
 +    }
++
 +    os.makedirs(os.path.dirname(os.path.join(element_path, element_name)), exist_ok=True)
 +    _yaml.dump(element, os.path.join(element_path, element_name))
++
 +    # First we check that we get the "Silly message"
 +    result = cli.run(project=project, args=["build", element_name])
 +    result.assert_success()
 +    assert 'echo "Silly message"' in result.stderr
++
 +    # Let's now build it again, but with --message-lines 0
 +    cli.remove_artifact_from_cache(project, element_name)
 +    result = cli.run(project=project, args=["--message-lines", "0",
 +                                            "build", element_name])
 +    result.assert_success()
 +    assert "Message contains " not in result.stderr
++
++
 +@pytest.mark.integration
 +@pytest.mark.datafiles(DATA_DIR)
 +@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
 +def test_disable_error_lines(cli, tmpdir, datafiles):
 +    project = os.path.join(datafiles.dirname, datafiles.basename)
 +    element_path = os.path.join(project, 'elements')
 +    element_name = 'message.bst'
++
 +    element = {
 +        'kind': 'manual',
 +        'depends': [{
 +            'filename': 'base.bst'
 +        }],
 +        'config': {
 +            'build-commands':
 +            ['This is a syntax error > >'],
 +            'strip-commands': []
 +        }
 +    }
++
 +    os.makedirs(os.path.dirname(os.path.join(element_path, element_name)), exist_ok=True)
 +    _yaml.dump(element, os.path.join(element_path, element_name))
++
 +    # First we check that we get the syntax error
 +    result = cli.run(project=project, args=["--error-lines", "0",
 +                                            "build", element_name])
 +    result.assert_main_error(ErrorDomain.STREAM, None)
 +    assert "This is a syntax error" in result.stderr
++
 +    # Let's now build it again, but with --error-lines 0
 +    cli.remove_artifact_from_cache(project, element_name)
 +    result = cli.run(project=project, args=["--error-lines", "0",
 +                                            "build", element_name])
 +    result.assert_main_error(ErrorDomain.STREAM, None)
 +    assert "Printing the last" not in result.stderr

tests/testutils/python_repo.py

 -from setuptools.sandbox import run_setup
  import os
  import pytest
  import re
  import shutil
 +import subprocess
  SETUP_TEMPLATE = '''\
@@ -88,7 +88,9 @@ def generate_pip_package(tmpdir, pypi, name, version='0.1'):
          f.write(INIT_TEMPLATE.format(name=name))
      os.chmod(main_file, 0o644)
 -    run_setup(setup_file, ['sdist'])
 +    # Run sdist with a fresh process
 +    p = subprocess.run(['python3', 'setup.py', 'sdist'], cwd=tmpdir)
 +    assert p.returncode == 0
      # create directory for this package in pypi resulting in a directory
      # tree resembling the following structure:

tests/testutils/runcli.py

@@ -245,8 +245,14 @@ class Cli():
      def remove_artifact_from_cache(self, project, element_name,
                                     *, cache_dir=None):
 +        # Read configuration to figure out where artifacts are stored
          if not cache_dir:
 -            cache_dir = os.path.join(project, 'cache', 'artifacts')
 +            default = os.path.join(project, 'cache', 'artifacts')
++
 +            if self.config is not None:
 +                cache_dir = self.config.get('artifactdir', default)
 +            else:
 +                cache_dir = default
          cache_dir = os.path.join(cache_dir, 'cas', 'refs', 'heads')

[Notes] [Git][BuildStream/buildstream][tlater/message-lines] 6 commits: tests/testutils/python_repo.py: Use subprocess to run sdist

Tristan Van Berkom pushed to branch tlater/message-lines at BuildStream / buildstream

Commits:

5 changed files:

Changes: