[Notes] [Git][BuildStream/buildstream][willsalmon/log_formating] 11 commits: requirements: Add roaringbitmap to the requirements

[Will Salmon <gitlab mg gitlab com>]

Title: GitLab

Will Salmon pushed to branch willsalmon/log_formating at BuildStream / buildstream

Commits:

• 9c94e8e5
  by Daniel Silverstone at 2019-02-05T13:15:24Z

  requirements: Add roaringbitmap to the requirements
  
  In order to support use of roaring bitmaps in the loader, we need to
  depend on it here.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 7682ef49
  by Daniel Silverstone at 2019-02-05T13:15:24Z

  loadelement.py: Use roaring bitmaps in dep cache
  
  The dependency cache in LoadElement causes the peak RAM consumption of
  the loader to be exceedingly large, upwards of 25GB for a test of a
  Debian stack.  By switching from the old dict cache to a roaring bitmap
  based cache, we reduce the cost of the loader cache in RAM terms to
  around 5.5GB for the same stack, which makes it plausible for the short
  term.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 0e471144
  by Benjamin Schubert at 2019-02-05T15:45:37Z

  Merge branch 'danielsilverstone-ct/roaring-bitmaps' into 'master'
  
  Switch to roaring bitmaps for the loader dependency caches
  
  See merge request BuildStream/buildstream!1128

• f17ea6a6
  by Jonathan Maw at 2019-02-06T10:32:05Z

  gitlab-ci: Add commands to be run by WSL runners
  

• d4d2f897
  by Jonathan Maw at 2019-02-06T10:45:34Z

  tests: Skip tests that use sandboxes on WSL
  

• f4c8a0ec
  by Jonathan Maw at 2019-02-06T10:45:34Z

  tests: Fix test failures caused by unix sockets' path length limit
  
  The unix specification for unix sockets only allocates ~100 characters
  for the length of the path. This may be longer than the path for a CI
  runner, or the path to a user's buildstream directory.
  

• 256bcde5
  by Jonathan Maw at 2019-02-06T15:29:18Z

  Merge branch 'jonathan/wsl-tests' into 'master'
  
  Add pre-merge tests that use a WSL runner
  
  Closes #852
  
  See merge request BuildStream/buildstream!1108

• 335ef2b8
  by Daniel Silverstone at 2019-02-06T17:30:13Z

  loadelement.py: Switch from roaringbitmap to pyroaring
  
  The licence for roaringbitmap was not compatible.  pyroaring is
  a similar project which is MIT licensed and is slightly better on
  memory too.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 8f90be91
  by Benjamin Schubert at 2019-02-06T18:36:17Z

  Update testing images to include c++ compiler
  

• 1a75b252
  by Benjamin Schubert at 2019-02-06T19:34:09Z

  Merge branch 'danielsilverstone-ct/other-roaring' into 'master'
  
  loadelement.py: Switch from roaringbitmap to pyroaring
  
  Closes #899
  
  See merge request BuildStream/buildstream!1133

• 7285e87e
  by William Salmon at 2019-02-07T09:42:36Z

  Add more log formatting options
  
  The 'wallclock-us' is a option that we thought we already had.
  

27 changed files:

• .gitlab-ci.yml
• buildstream/_frontend/widget.py
• buildstream/_loader/loadelement.py
• requirements/requirements.in
• requirements/requirements.txt
• tests/frontend/logging.py
• tests/integration/autotools.py
• tests/integration/build-tree.py
• tests/integration/build-uid.py
• tests/integration/cachedfail.py
• tests/integration/cmake.py
• tests/integration/compose.py
• tests/integration/make.py
• tests/integration/manual.py
• tests/integration/messages.py
• tests/integration/pip_element.py
• tests/integration/pip_source.py
• tests/integration/pullbuildtrees.py
• tests/integration/script.py
• tests/integration/shell.py
• tests/integration/sockets.py
• tests/integration/source-determinism.py
• tests/integration/stack.py
• tests/integration/symlinks.py
• tests/integration/workspace.py
• tests/testutils/filetypegenerator.py
• tests/testutils/site.py

Changes:

.gitlab-ci.yml

---

1		-image: buildstream/testsuite-debian:9-5da27168-32c47d1c
	1	+image: registry.gitlab.com/buildstream/buildstream-docker-images/testsuite-debian:9-master-46405991
2	2
3	3	cache:
4	4	key: "$CI_JOB_NAME-"
...	...	@@ -46,23 +46,23 @@ variables:
46	46	- .coverage-reports
47	47
48	48	tests-debian-9:
49		- image: buildstream/testsuite-debian:9-5da27168-32c47d1c
	49	+ image: registry.gitlab.com/buildstream/buildstream-docker-images/testsuite-debian:9-master-46405991
50	50	<<: *tests
51	51
52	52	tests-fedora-27:
53		- image: buildstream/testsuite-fedora:27-5da27168-32c47d1c
	53	+ image: registry.gitlab.com/buildstream/buildstream-docker-images/testsuite-fedora:27-master-46405991
54	54	<<: *tests
55	55
56	56	tests-fedora-28:
57		- image: buildstream/testsuite-fedora:28-5da27168-32c47d1c
	57	+ image: registry.gitlab.com/buildstream/buildstream-docker-images/testsuite-fedora:28-master-46405991
58	58	<<: *tests
59	59
60	60	tests-ubuntu-18.04:
61		- image: buildstream/testsuite-ubuntu:18.04-5da27168-32c47d1c
	61	+ image: registry.gitlab.com/buildstream/buildstream-docker-images/testsuite-ubuntu:18.04-master-46405991
62	62	<<: *tests
63	63
64	64	tests-python-3.7-stretch:
65		- image: buildstream/testsuite-python:3.7-stretch-a60f0c39
	65	+ image: registry.gitlab.com/buildstream/buildstream-docker-images/testsuite-python:3.7-stretch-master-46405991
66	66	<<: *tests
67	67
68	68	variables:
...	...	@@ -73,10 +73,10 @@ tests-python-3.7-stretch:
73	73
74	74	tests-centos-7.6:
75	75	<<: *tests
76		- image: buildstream/testsuite-centos:7.6-5da27168-32c47d1c
	76	+ image: registry.gitlab.com/buildstream/buildstream-docker-images/testsuite-centos:7.6.1810-master-46405991
77	77
78	78	overnight-fedora-28-aarch64:
79		- image: buildstream/testsuite-fedora:aarch64-28-5da27168-32c47d1c
	79	+ image: registry.gitlab.com/buildstream/buildstream-docker-images/testsuite-fedora:aarch64-28-master-46405991
80	80	tags:
81	81	- aarch64
82	82	<<: *tests
...	...	@@ -95,7 +95,7 @@ overnight-fedora-28-aarch64:
95	95	tests-unix:
96	96	# Use fedora here, to a) run a test on fedora and b) ensure that we
97	97	# can get rid of ostree - this is not possible with debian-8
98		- image: buildstream/testsuite-fedora:27-5da27168-32c47d1c
	98	+ image: registry.gitlab.com/buildstream/buildstream-docker-images/testsuite-fedora:27-master-46405991
99	99	<<: *tests
100	100	variables:
101	101	BST_FORCE_BACKEND: "unix"
...	...	@@ -105,7 +105,7 @@ tests-unix:
105	105	# We remove the Bubblewrap and OSTree packages here so that we catch any
106	106	# codepaths that try to use them. Removing OSTree causes fuse-libs to
107	107	# disappear unless we mark it as user-installed.
108		- - dnf mark install fuse-libs
	108	+ - dnf mark install fuse-libs systemd-udev
109	109	- dnf erase -y bubblewrap ostree
110	110
111	111	# Since the unix platform is required to run as root, no user change required
...	...	@@ -113,14 +113,14 @@ tests-unix:
113	113
114	114	tests-fedora-missing-deps:
115	115	# Ensure that tests behave nicely while missing bwrap and ostree
116		- image: buildstream/testsuite-fedora:28-5da27168-32c47d1c
	116	+ image: registry.gitlab.com/buildstream/buildstream-docker-images/testsuite-fedora:28-master-46405991
117	117	<<: *tests
118	118
119	119	script:
120	120	# We remove the Bubblewrap and OSTree packages here so that we catch any
121	121	# codepaths that try to use them. Removing OSTree causes fuse-libs to
122	122	# disappear unless we mark it as user-installed.
123		- - dnf mark install fuse-libs
	123	+ - dnf mark install fuse-libs systemd-udev
124	124	- dnf erase -y bubblewrap ostree
125	125
126	126	- useradd -Um buildstream
...	...	@@ -132,7 +132,7 @@ tests-fedora-update-deps:
132	132	# Check if the tests pass after updating requirements to their latest
133	133	# allowed version.
134	134	allow_failure: true
135		- image: buildstream/testsuite-fedora:28-5da27168-32c47d1c
	135	+ image: registry.gitlab.com/buildstream/buildstream-docker-images/testsuite-fedora:28-master-46405991
136	136	<<: *tests
137	137
138	138	script:
...	...	@@ -157,6 +157,20 @@ lint:
157	157	except:
158	158	- schedules
159	159
	160	+tests-wsl:
	161	+ stage: test
	162	+ variables:
	163	+ LC_ALL: C.UTF-8
	164	+ LANG: C.UTF-8
	165	+ tags:
	166	+ - wsl
	167	+ before_script:
	168	+ - mount
	169	+ - df -h
	170	+
	171	+ script:
	172	+ - "${TEST_COMMAND}"
	173	+
160	174	# Automatically build documentation for every commit, we want to know
161	175	# if building documentation fails even if we're not deploying it.
162	176	docs:

buildstream/_frontend/widget.py

---

...	...	@@ -94,12 +94,24 @@ class FixedText(Widget):
94	94
95	95	# Used to add the wallclock time this message was created at
96	96	class WallclockTime(Widget):
	97	+ def __init__(self, context, content_profile, format_profile, output_format=False):
	98	+ self._output_format = output_format
	99	+ super(WallclockTime, self).__init__(context, content_profile, format_profile)
	100	+
97	101	def render(self, message):
	102	+
98	103	fields = [self.content_profile.fmt("{:02d}".format(x)) for x in
99	104	[message.creation_time.hour,
100	105	message.creation_time.minute,
101		- message.creation_time.second]]
102		- return self.format_profile.fmt(":").join(fields)
	106	+ message.creation_time.second,
	107	+ ]
	108	+ ]
	109	+ text = self.format_profile.fmt(":").join(fields)
	110	+
	111	+ if self._output_format == 'us':
	112	+ text += self.content_profile.fmt(".{:06d}".format(message.creation_time.microsecond))
	113	+
	114	+ return text
103	115
104	116
105	117	# A widget for rendering the debugging column
...	...	@@ -326,6 +338,7 @@ class LogLine(Widget):
326	338	"elapsed": TimeCode(context, content_profile, format_profile, microseconds=False),
327	339	"elapsed-us": TimeCode(context, content_profile, format_profile, microseconds=True),
328	340	"wallclock": WallclockTime(context, content_profile, format_profile),
	341	+ "wallclock-us": WallclockTime(context, content_profile, format_profile, output_format='us'),
329	342	"key": CacheKey(context, content_profile, format_profile, err_profile),
330	343	"element": ElementName(context, content_profile, format_profile),
331	344	"action": TypeName(context, content_profile, format_profile),

buildstream/_loader/loadelement.py

---

...	...	@@ -19,6 +19,9 @@
19	19
20	20	# System imports
21	21	from collections.abc import Mapping
	22	+from itertools import count
	23	+
	24	+from pyroaring import BitMap, FrozenBitMap # pylint: disable=no-name-in-module
22	25
23	26	# BuildStream toplevel imports
24	27	from .._exceptions import LoadError, LoadErrorReason
...	...	@@ -54,6 +57,8 @@ class LoadElement():
54	57	self.element = element
55	58	self.dep_type = dep_type
56	59
	60	+ _counter = count()
	61	+
57	62	def __init__(self, node, filename, loader):
58	63
59	64	#
...	...	@@ -63,6 +68,7 @@ class LoadElement():
63	68	self.name = filename # The element name
64	69	self.full_name = None # The element full name (with associated junction)
65	70	self.deps = None # The list of Dependency objects
	71	+ self.node_id = next(self._counter)
66	72
67	73	#
68	74	# Private members
...	...	@@ -107,7 +113,7 @@ class LoadElement():
107	113	#
108	114	def depends(self, other):
109	115	self._ensure_depends_cache()
110		- return self._dep_cache.get(other.full_name) is not None
	116	+ return other.node_id in self._dep_cache
111	117
112	118	###########################################
113	119	# Private Methods #
...	...	@@ -117,7 +123,8 @@ class LoadElement():
117	123	if self._dep_cache:
118	124	return
119	125
120		- self._dep_cache = {}
	126	+ self._dep_cache = BitMap()
	127	+
121	128	for dep in self.dependencies:
122	129	elt = dep.element
123	130
...	...	@@ -125,11 +132,13 @@ class LoadElement():
125	132	elt._ensure_depends_cache()
126	133
127	134	# We depend on this element
128		- self._dep_cache[elt.full_name] = True
	135	+ self._dep_cache.add(elt.node_id)
129	136
130	137	# And we depend on everything this element depends on
131	138	self._dep_cache.update(elt._dep_cache)
132	139
	140	+ self._dep_cache = FrozenBitMap(self._dep_cache)
	141	+
133	142
134	143	# _extract_depends_from_node():
135	144	#

requirements/requirements.in

---

...	...	@@ -13,3 +13,4 @@ psutil
13	13	# See issues #571 and #790.
14	14	ruamel.yaml >= 0.15.41, < 0.15.52
15	15	setuptools
	16	+pyroaring

requirements/requirements.txt

---

...	...	@@ -13,6 +13,7 @@ psutil==5.4.8
13	13	# See issues #571 and #790.
14	14	ruamel.yaml==0.15.51
15	15	setuptools==39.0.1
	16	+pyroaring==0.2.6
16	17	## The following requirements were added by pip freeze:
17	18	MarkupSafe==1.1.0
18	19	six==1.12.0

tests/frontend/logging.py

---

...	...	@@ -52,7 +52,8 @@ def test_custom_logging(cli, tmpdir, datafiles):
52	52	element_path = os.path.join(project, 'elements')
53	53	element_name = 'fetch-test-git.bst'
54	54
55		- custom_log_format = '%{elapsed},%{elapsed-us},%{wallclock},%{key},%{element},%{action},%{message}'
	55	+ custom_log_format = ('%{elapsed},%{elapsed-us},%{wallclock},%{wallclock-us},'
	56	+ '%{key},%{element},%{action},%{message}')
56	57	user_config = {'logging': {'message-format': custom_log_format}}
57	58	cli.configure(user_config)
58	59
...	...	@@ -77,7 +78,8 @@ def test_custom_logging(cli, tmpdir, datafiles):
77	78	result = cli.run(project=project, args=['source', 'fetch', element_name])
78	79	result.assert_success()
79	80
80		- m = re.search(r"\d\d:\d\d:\d\d,\d\d:\d\d:\d\d.\d{6},\d\d:\d\d:\d\d,\s*,.*,SUCCESS,Checking sources", result.stderr)
	81	+ m = re.search(r"\d\d:\d\d:\d\d,\d\d:\d\d:\d\d.\d{6},\d\d:\d\d:\d\d,\d\d:\d\d:\d\d.\d{6}\s*,.*"
	82	+ r",SUCCESS,Checking sources", result.stderr)
81	83	assert(m is not None)
82	84
83	85

tests/integration/autotools.py

---

...	...	@@ -3,7 +3,7 @@ import pytest
3	3
4	4	from tests.testutils import cli_integration as cli
5	5	from tests.testutils.integration import assert_contains
6		-from tests.testutils.site import HAVE_BWRAP, IS_LINUX
	6	+from tests.testutils.site import HAVE_SANDBOX
7	7
8	8
9	9	pytestmark = pytest.mark.integration
...	...	@@ -19,7 +19,7 @@ DATA_DIR = os.path.join(
19	19	# amhello project for this.
20	20	@pytest.mark.integration
21	21	@pytest.mark.datafiles(DATA_DIR)
22		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	22	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
23	23	def test_autotools_build(cli, tmpdir, datafiles):
24	24	project = os.path.join(datafiles.dirname, datafiles.basename)
25	25	checkout = os.path.join(cli.directory, 'checkout')
...	...	@@ -42,7 +42,7 @@ def test_autotools_build(cli, tmpdir, datafiles):
42	42	# amhello project for this.
43	43	@pytest.mark.integration
44	44	@pytest.mark.datafiles(DATA_DIR)
45		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	45	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
46	46	def test_autotools_confroot_build(cli, tmpdir, datafiles):
47	47	project = os.path.join(datafiles.dirname, datafiles.basename)
48	48	checkout = os.path.join(cli.directory, 'checkout')
...	...	@@ -63,7 +63,7 @@ def test_autotools_confroot_build(cli, tmpdir, datafiles):
63	63
64	64	# Test running an executable built with autotools
65	65	@pytest.mark.datafiles(DATA_DIR)
66		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	66	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
67	67	def test_autotools_run(cli, tmpdir, datafiles):
68	68	project = os.path.join(datafiles.dirname, datafiles.basename)
69	69	element_name = 'autotools/amhello.bst'

tests/integration/build-tree.py

---

...	...	@@ -3,7 +3,7 @@ import pytest
3	3	import shutil
4	4
5	5	from tests.testutils import cli, cli_integration, create_artifact_share
6		-from tests.testutils.site import HAVE_BWRAP, IS_LINUX
	6	+from tests.testutils.site import HAVE_SANDBOX
7	7	from buildstream._exceptions import ErrorDomain
8	8
9	9
...	...	@@ -17,7 +17,7 @@ DATA_DIR = os.path.join(
17	17
18	18
19	19	@pytest.mark.datafiles(DATA_DIR)
20		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	20	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
21	21	def test_buildtree_staged(cli_integration, tmpdir, datafiles):
22	22	# We can only test the non interacitve case
23	23	# The non interactive case defaults to not using buildtrees
...	...	@@ -35,7 +35,7 @@ def test_buildtree_staged(cli_integration, tmpdir, datafiles):
35	35
36	36
37	37	@pytest.mark.datafiles(DATA_DIR)
38		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	38	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
39	39	def test_buildtree_staged_forced_true(cli_integration, tmpdir, datafiles):
40	40	# Test that if we ask for a build tree it is there.
41	41	project = os.path.join(datafiles.dirname, datafiles.basename)
...	...	@@ -52,7 +52,7 @@ def test_buildtree_staged_forced_true(cli_integration, tmpdir, datafiles):
52	52
53	53
54	54	@pytest.mark.datafiles(DATA_DIR)
55		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	55	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
56	56	def test_buildtree_staged_if_available(cli_integration, tmpdir, datafiles):
57	57	# Test that a build tree can be correctly detected.
58	58	project = os.path.join(datafiles.dirname, datafiles.basename)
...	...	@@ -69,7 +69,7 @@ def test_buildtree_staged_if_available(cli_integration, tmpdir, datafiles):
69	69
70	70
71	71	@pytest.mark.datafiles(DATA_DIR)
72		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	72	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
73	73	def test_buildtree_staged_forced_false(cli_integration, tmpdir, datafiles):
74	74	# Test that if we ask not to have a build tree it is not there
75	75	project = os.path.join(datafiles.dirname, datafiles.basename)
...	...	@@ -87,7 +87,7 @@ def test_buildtree_staged_forced_false(cli_integration, tmpdir, datafiles):
87	87
88	88
89	89	@pytest.mark.datafiles(DATA_DIR)
90		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	90	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
91	91	def test_buildtree_from_failure(cli_integration, tmpdir, datafiles):
92	92	# Test that we can use a build tree after a failure
93	93	project = os.path.join(datafiles.dirname, datafiles.basename)
...	...	@@ -108,7 +108,7 @@ def test_buildtree_from_failure(cli_integration, tmpdir, datafiles):
108	108	# Check that build shells work when pulled from a remote cache
109	109	# This is to roughly simulate remote execution
110	110	@pytest.mark.datafiles(DATA_DIR)
111		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	111	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
112	112	def test_buildtree_pulled(cli, tmpdir, datafiles):
113	113	project = os.path.join(datafiles.dirname, datafiles.basename)
114	114	element_name = 'build-shell/buildtree.bst'
...	...	@@ -143,7 +143,7 @@ def test_buildtree_pulled(cli, tmpdir, datafiles):
143	143
144	144	# This test checks for correct behaviour if a buildtree is not present in the local cache.
145	145	@pytest.mark.datafiles(DATA_DIR)
146		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	146	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
147	147	def test_buildtree_options(cli, tmpdir, datafiles):
148	148	project = os.path.join(datafiles.dirname, datafiles.basename)
149	149	element_name = 'build-shell/buildtree.bst'

tests/integration/build-uid.py

---

...	...	@@ -5,7 +5,7 @@ from buildstream import _yaml
5	5
6	6	from tests.testutils import cli_integration as cli
7	7	from tests.testutils.integration import assert_contains
8		-from tests.testutils.site import HAVE_BWRAP, IS_LINUX
	8	+from tests.testutils.site import HAVE_BWRAP, IS_LINUX, HAVE_SANDBOX
9	9
10	10
11	11	pytestmark = pytest.mark.integration
...	...	@@ -55,7 +55,7 @@ def test_build_uid_in_project(cli, tmpdir, datafiles):
55	55
56	56
57	57	@pytest.mark.datafiles(DATA_DIR)
58		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	58	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
59	59	def test_build_uid_default(cli, tmpdir, datafiles):
60	60	project = os.path.join(datafiles.dirname, datafiles.basename)
61	61	checkout = os.path.join(cli.directory, 'checkout')

tests/integration/cachedfail.py

---

...	...	@@ -7,7 +7,7 @@ from buildstream._exceptions import ErrorDomain
7	7	from conftest import clean_platform_cache
8	8
9	9	from tests.testutils import cli_integration as cli, create_artifact_share
10		-from tests.testutils.site import HAVE_BWRAP, IS_LINUX
	10	+from tests.testutils.site import HAVE_BWRAP, IS_LINUX, HAVE_SANDBOX
11	11
12	12
13	13	pytestmark = pytest.mark.integration
...	...	@@ -20,7 +20,7 @@ DATA_DIR = os.path.join(
20	20
21	21
22	22	@pytest.mark.datafiles(DATA_DIR)
23		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	23	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
24	24	def test_build_checkout_cached_fail(cli, tmpdir, datafiles):
25	25	project = os.path.join(datafiles.dirname, datafiles.basename)
26	26	element_path = os.path.join(project, 'elements', 'element.bst')
...	...	@@ -64,7 +64,7 @@ def test_build_checkout_cached_fail(cli, tmpdir, datafiles):
64	64
65	65
66	66	@pytest.mark.datafiles(DATA_DIR)
67		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	67	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
68	68	def test_build_depend_on_cached_fail(cli, tmpdir, datafiles):
69	69	project = os.path.join(datafiles.dirname, datafiles.basename)
70	70	dep_path = os.path.join(project, 'elements', 'dep.bst')
...	...	@@ -123,7 +123,7 @@ def test_build_depend_on_cached_fail(cli, tmpdir, datafiles):
123	123	assert cli.get_element_state(project, 'target.bst') == 'waiting'
124	124
125	125
126		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	126	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
127	127	@pytest.mark.datafiles(DATA_DIR)
128	128	@pytest.mark.parametrize("on_error", ("continue", "quit"))
129	129	def test_push_cached_fail(cli, tmpdir, datafiles, on_error):

tests/integration/cmake.py

---

...	...	@@ -3,7 +3,7 @@ import pytest
3	3
4	4	from tests.testutils import cli_integration as cli
5	5	from tests.testutils.integration import assert_contains
6		-from tests.testutils.site import HAVE_BWRAP, IS_LINUX
	6	+from tests.testutils.site import HAVE_SANDBOX
7	7
8	8
9	9	pytestmark = pytest.mark.integration
...	...	@@ -16,7 +16,7 @@ DATA_DIR = os.path.join(
16	16
17	17
18	18	@pytest.mark.datafiles(DATA_DIR)
19		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	19	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
20	20	def test_cmake_build(cli, tmpdir, datafiles):
21	21	project = os.path.join(datafiles.dirname, datafiles.basename)
22	22	checkout = os.path.join(cli.directory, 'checkout')
...	...	@@ -32,7 +32,7 @@ def test_cmake_build(cli, tmpdir, datafiles):
32	32
33	33
34	34	@pytest.mark.datafiles(DATA_DIR)
35		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	35	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
36	36	def test_cmake_confroot_build(cli, tmpdir, datafiles):
37	37	project = os.path.join(datafiles.dirname, datafiles.basename)
38	38	checkout = os.path.join(cli.directory, 'checkout')
...	...	@@ -48,7 +48,7 @@ def test_cmake_confroot_build(cli, tmpdir, datafiles):
48	48
49	49
50	50	@pytest.mark.datafiles(DATA_DIR)
51		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	51	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
52	52	def test_cmake_run(cli, tmpdir, datafiles):
53	53	project = os.path.join(datafiles.dirname, datafiles.basename)
54	54	element_name = 'cmake/cmakehello.bst'

tests/integration/compose.py

---

...	...	@@ -7,7 +7,7 @@ from buildstream import _yaml
7	7
8	8	from tests.testutils import cli_integration as cli
9	9	from tests.testutils.integration import walk_dir
10		-from tests.testutils.site import HAVE_BWRAP, IS_LINUX
	10	+from tests.testutils.site import HAVE_SANDBOX
11	11
12	12
13	13	pytestmark = pytest.mark.integration
...	...	@@ -75,7 +75,7 @@ def create_compose_element(name, path, config={}):
75	75	'/usr/share/doc', '/usr/share/doc/amhello',
76	76	'/usr/share/doc/amhello/README'])
77	77	])
78		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	78	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
79	79	def test_compose_include(cli, tmpdir, datafiles, include_domains,
80	80	exclude_domains, expected):
81	81	project = os.path.join(datafiles.dirname, datafiles.basename)

tests/integration/make.py

---

...	...	@@ -3,7 +3,7 @@ import pytest
3	3
4	4	from tests.testutils import cli_integration as cli
5	5	from tests.testutils.integration import assert_contains
6		-from tests.testutils.site import HAVE_BWRAP, IS_LINUX
	6	+from tests.testutils.site import HAVE_SANDBOX
7	7
8	8
9	9	pytestmark = pytest.mark.integration
...	...	@@ -19,7 +19,7 @@ DATA_DIR = os.path.join(
19	19	# makehello project for this.
20	20	@pytest.mark.integration
21	21	@pytest.mark.datafiles(DATA_DIR)
22		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	22	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
23	23	def test_make_build(cli, tmpdir, datafiles):
24	24	project = os.path.join(datafiles.dirname, datafiles.basename)
25	25	checkout = os.path.join(cli.directory, 'checkout')
...	...	@@ -37,7 +37,7 @@ def test_make_build(cli, tmpdir, datafiles):
37	37
38	38	# Test running an executable built with make
39	39	@pytest.mark.datafiles(DATA_DIR)
40		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	40	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
41	41	def test_make_run(cli, tmpdir, datafiles):
42	42	project = os.path.join(datafiles.dirname, datafiles.basename)
43	43	element_name = 'make/makehello.bst'

tests/integration/manual.py

---

...	...	@@ -4,7 +4,7 @@ import pytest
4	4	from buildstream import _yaml
5	5
6	6	from tests.testutils import cli_integration as cli
7		-from tests.testutils.site import HAVE_BWRAP, IS_LINUX
	7	+from tests.testutils.site import HAVE_SANDBOX
8	8
9	9
10	10	pytestmark = pytest.mark.integration
...	...	@@ -32,7 +32,7 @@ def create_manual_element(name, path, config, variables, environment):
32	32
33	33
34	34	@pytest.mark.datafiles(DATA_DIR)
35		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	35	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
36	36	def test_manual_element(cli, tmpdir, datafiles):
37	37	project = os.path.join(datafiles.dirname, datafiles.basename)
38	38	checkout = os.path.join(cli.directory, 'checkout')
...	...	@@ -66,7 +66,7 @@ strip
66	66
67	67
68	68	@pytest.mark.datafiles(DATA_DIR)
69		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	69	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
70	70	def test_manual_element_environment(cli, tmpdir, datafiles):
71	71	project = os.path.join(datafiles.dirname, datafiles.basename)
72	72	checkout = os.path.join(cli.directory, 'checkout')
...	...	@@ -96,7 +96,7 @@ def test_manual_element_environment(cli, tmpdir, datafiles):
96	96
97	97
98	98	@pytest.mark.datafiles(DATA_DIR)
99		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	99	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
100	100	def test_manual_element_noparallel(cli, tmpdir, datafiles):
101	101	project = os.path.join(datafiles.dirname, datafiles.basename)
102	102	checkout = os.path.join(cli.directory, 'checkout')
...	...	@@ -131,7 +131,7 @@ def test_manual_element_noparallel(cli, tmpdir, datafiles):
131	131
132	132
133	133	@pytest.mark.datafiles(DATA_DIR)
134		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	134	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
135	135	def test_manual_element_logging(cli, tmpdir, datafiles):
136	136	project = os.path.join(datafiles.dirname, datafiles.basename)
137	137	checkout = os.path.join(cli.directory, 'checkout')

tests/integration/messages.py

---

...	...	@@ -24,7 +24,7 @@ from buildstream import _yaml
24	24	from buildstream._exceptions import ErrorDomain
25	25
26	26	from tests.testutils import cli_integration as cli
27		-from tests.testutils.site import HAVE_BWRAP, IS_LINUX
	27	+from tests.testutils.site import HAVE_SANDBOX
28	28
29	29
30	30	pytestmark = pytest.mark.integration
...	...	@@ -39,7 +39,7 @@ DATA_DIR = os.path.join(
39	39
40	40	@pytest.mark.integration
41	41	@pytest.mark.datafiles(DATA_DIR)
42		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	42	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
43	43	def test_disable_message_lines(cli, tmpdir, datafiles):
44	44	project = os.path.join(datafiles.dirname, datafiles.basename)
45	45	element_path = os.path.join(project, 'elements')
...	...	@@ -75,7 +75,7 @@ def test_disable_message_lines(cli, tmpdir, datafiles):
75	75
76	76	@pytest.mark.integration
77	77	@pytest.mark.datafiles(DATA_DIR)
78		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	78	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
79	79	def test_disable_error_lines(cli, tmpdir, datafiles):
80	80	project = os.path.join(datafiles.dirname, datafiles.basename)
81	81	element_path = os.path.join(project, 'elements')

tests/integration/pip_element.py

---

...	...	@@ -6,7 +6,7 @@ from buildstream import _yaml
6	6
7	7	from tests.testutils import cli_integration as cli
8	8	from tests.testutils.integration import assert_contains
9		-from tests.testutils.site import HAVE_BWRAP, IS_LINUX
	9	+from tests.testutils.site import HAVE_SANDBOX
10	10
11	11
12	12	pytestmark = pytest.mark.integration
...	...	@@ -19,7 +19,7 @@ DATA_DIR = os.path.join(
19	19
20	20
21	21	@pytest.mark.datafiles(DATA_DIR)
22		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	22	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
23	23	def test_pip_build(cli, tmpdir, datafiles):
24	24	project = os.path.join(datafiles.dirname, datafiles.basename)
25	25	checkout = os.path.join(cli.directory, 'checkout')
...	...	@@ -56,7 +56,7 @@ def test_pip_build(cli, tmpdir, datafiles):
56	56
57	57	# Test running an executable built with pip
58	58	@pytest.mark.datafiles(DATA_DIR)
59		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	59	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
60	60	def test_pip_run(cli, tmpdir, datafiles):
61	61	# Create and build our test element
62	62	test_pip_build(cli, tmpdir, datafiles)

tests/integration/pip_source.py

---

...	...	@@ -6,7 +6,7 @@ from buildstream import _yaml
6	6	from tests.testutils import cli_integration as cli
7	7	from tests.testutils.python_repo import setup_pypi_repo
8	8	from tests.testutils.integration import assert_contains
9		-from tests.testutils.site import HAVE_BWRAP, IS_LINUX
	9	+from tests.testutils.site import HAVE_SANDBOX
10	10
11	11
12	12	pytestmark = pytest.mark.integration
...	...	@@ -73,7 +73,7 @@ def test_pip_source_import(cli, tmpdir, datafiles, setup_pypi_repo):
73	73
74	74
75	75	@pytest.mark.datafiles(DATA_DIR)
76		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	76	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
77	77	def test_pip_source_build(cli, tmpdir, datafiles, setup_pypi_repo):
78	78	project = os.path.join(datafiles.dirname, datafiles.basename)
79	79	element_path = os.path.join(project, 'elements')

tests/integration/pullbuildtrees.py

---

...	...	@@ -4,7 +4,7 @@ import pytest
4	4
5	5	from tests.testutils import cli, cli_integration as cli2, create_artifact_share
6	6	from tests.testutils.integration import assert_contains
7		-from tests.testutils.site import HAVE_BWRAP, IS_LINUX
	7	+from tests.testutils.site import HAVE_SANDBOX
8	8	from buildstream._exceptions import ErrorDomain, LoadErrorReason
9	9
10	10
...	...	@@ -31,7 +31,7 @@ def default_state(cli, tmpdir, share):
31	31	# directory of an element.
32	32	@pytest.mark.integration
33	33	@pytest.mark.datafiles(DATA_DIR)
34		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	34	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
35	35	def test_pullbuildtrees(cli2, tmpdir, datafiles):
36	36	project = os.path.join(datafiles.dirname, datafiles.basename)
37	37	element_name = 'autotools/amhello.bst'

tests/integration/script.py

---

...	...	@@ -4,7 +4,7 @@ import pytest
4	4	from buildstream import _yaml
5	5
6	6	from tests.testutils import cli_integration as cli
7		-from tests.testutils.site import HAVE_BWRAP, IS_LINUX
	7	+from tests.testutils.site import HAVE_SANDBOX
8	8
9	9
10	10	pytestmark = pytest.mark.integration
...	...	@@ -31,7 +31,7 @@ def create_script_element(name, path, config={}, variables={}):
31	31
32	32
33	33	@pytest.mark.datafiles(DATA_DIR)
34		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	34	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
35	35	def test_script(cli, tmpdir, datafiles):
36	36	project = os.path.join(datafiles.dirname, datafiles.basename)
37	37	checkout = os.path.join(cli.directory, 'checkout')
...	...	@@ -59,7 +59,7 @@ def test_script(cli, tmpdir, datafiles):
59	59
60	60
61	61	@pytest.mark.datafiles(DATA_DIR)
62		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	62	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
63	63	def test_script_root(cli, tmpdir, datafiles):
64	64	project = os.path.join(datafiles.dirname, datafiles.basename)
65	65	checkout = os.path.join(cli.directory, 'checkout')
...	...	@@ -91,7 +91,7 @@ def test_script_root(cli, tmpdir, datafiles):
91	91
92	92
93	93	@pytest.mark.datafiles(DATA_DIR)
94		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	94	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
95	95	def test_script_no_root(cli, tmpdir, datafiles):
96	96	project = os.path.join(datafiles.dirname, datafiles.basename)
97	97	element_path = os.path.join(project, 'elements')
...	...	@@ -114,7 +114,7 @@ def test_script_no_root(cli, tmpdir, datafiles):
114	114
115	115
116	116	@pytest.mark.datafiles(DATA_DIR)
117		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	117	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
118	118	def test_script_cwd(cli, tmpdir, datafiles):
119	119	project = os.path.join(datafiles.dirname, datafiles.basename)
120	120	checkout = os.path.join(cli.directory, 'checkout')
...	...	@@ -145,7 +145,7 @@ def test_script_cwd(cli, tmpdir, datafiles):
145	145
146	146
147	147	@pytest.mark.datafiles(DATA_DIR)
148		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	148	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
149	149	def test_script_layout(cli, tmpdir, datafiles):
150	150	project = os.path.join(datafiles.dirname, datafiles.basename)
151	151	checkout = os.path.join(cli.directory, 'checkout')
...	...	@@ -164,7 +164,7 @@ def test_script_layout(cli, tmpdir, datafiles):
164	164
165	165
166	166	@pytest.mark.datafiles(DATA_DIR)
167		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	167	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
168	168	def test_regression_cache_corruption(cli, tmpdir, datafiles):
169	169	project = str(datafiles)
170	170	checkout_original = os.path.join(cli.directory, 'checkout-original')
...	...	@@ -194,7 +194,7 @@ def test_regression_cache_corruption(cli, tmpdir, datafiles):
194	194
195	195
196	196	@pytest.mark.datafiles(DATA_DIR)
197		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	197	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
198	198	def test_regression_tmpdir(cli, tmpdir, datafiles):
199	199	project = str(datafiles)
200	200	element_name = 'script/tmpdir.bst'
...	...	@@ -204,7 +204,7 @@ def test_regression_tmpdir(cli, tmpdir, datafiles):
204	204
205	205
206	206	@pytest.mark.datafiles(DATA_DIR)
207		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	207	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
208	208	def test_regression_cache_corruption_2(cli, tmpdir, datafiles):
209	209	project = str(datafiles)
210	210	checkout_original = os.path.join(cli.directory, 'checkout-original')

tests/integration/shell.py

---

...	...	@@ -5,7 +5,7 @@ from buildstream import _yaml
5	5	from buildstream._exceptions import ErrorDomain
6	6
7	7	from tests.testutils import cli_integration as cli
8		-from tests.testutils.site import HAVE_BWRAP, IS_LINUX
	8	+from tests.testutils.site import HAVE_SANDBOX
9	9
10	10
11	11	pytestmark = pytest.mark.integration
...	...	@@ -49,7 +49,7 @@ def execute_shell(cli, project, command, *, config=None, mount=None, element='ba
49	49	# Test running something through a shell, allowing it to find the
50	50	# executable
51	51	@pytest.mark.datafiles(DATA_DIR)
52		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	52	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
53	53	def test_shell(cli, tmpdir, datafiles):
54	54	project = os.path.join(datafiles.dirname, datafiles.basename)
55	55
...	...	@@ -60,7 +60,7 @@ def test_shell(cli, tmpdir, datafiles):
60	60
61	61	# Test running an executable directly
62	62	@pytest.mark.datafiles(DATA_DIR)
63		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	63	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
64	64	def test_executable(cli, tmpdir, datafiles):
65	65	project = os.path.join(datafiles.dirname, datafiles.basename)
66	66
...	...	@@ -72,7 +72,7 @@ def test_executable(cli, tmpdir, datafiles):
72	72	# Test shell environment variable explicit assignments
73	73	@pytest.mark.parametrize("animal", [("Horse"), ("Pony")])
74	74	@pytest.mark.datafiles(DATA_DIR)
75		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	75	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
76	76	def test_env_assign(cli, tmpdir, datafiles, animal):
77	77	project = os.path.join(datafiles.dirname, datafiles.basename)
78	78	expected = animal + '\n'
...	...	@@ -92,7 +92,7 @@ def test_env_assign(cli, tmpdir, datafiles, animal):
92	92	# Test shell environment variable explicit assignments with host env var expansion
93	93	@pytest.mark.parametrize("animal", [("Horse"), ("Pony")])
94	94	@pytest.mark.datafiles(DATA_DIR)
95		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	95	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
96	96	def test_env_assign_expand_host_environ(cli, tmpdir, datafiles, animal):
97	97	project = os.path.join(datafiles.dirname, datafiles.basename)
98	98	expected = 'The animal is: {}\n'.format(animal)
...	...	@@ -115,7 +115,7 @@ def test_env_assign_expand_host_environ(cli, tmpdir, datafiles, animal):
115	115	# when running an isolated shell
116	116	@pytest.mark.parametrize("animal", [("Horse"), ("Pony")])
117	117	@pytest.mark.datafiles(DATA_DIR)
118		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	118	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
119	119	def test_env_assign_isolated(cli, tmpdir, datafiles, animal):
120	120	project = os.path.join(datafiles.dirname, datafiles.basename)
121	121	result = execute_shell(cli, project, ['/bin/sh', '-c', 'echo ${ANIMAL}'], isolate=True, config={
...	...	@@ -133,7 +133,7 @@ def test_env_assign_isolated(cli, tmpdir, datafiles, animal):
133	133	# Test running an executable in a runtime with no shell (i.e., no
134	134	# /bin/sh)
135	135	@pytest.mark.datafiles(DATA_DIR)
136		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	136	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
137	137	def test_no_shell(cli, tmpdir, datafiles):
138	138	project = os.path.join(datafiles.dirname, datafiles.basename)
139	139	element_path = os.path.join(project, 'elements')
...	...	@@ -166,7 +166,7 @@ def test_no_shell(cli, tmpdir, datafiles):
166	166	# Test that bind mounts defined in project.conf work
167	167	@pytest.mark.parametrize("path", [("/etc/pony.conf"), ("/usr/share/pony/pony.txt")])
168	168	@pytest.mark.datafiles(DATA_DIR)
169		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	169	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
170	170	def test_host_files(cli, tmpdir, datafiles, path):
171	171	project = os.path.join(datafiles.dirname, datafiles.basename)
172	172	ponyfile = os.path.join(project, 'files', 'shell-mount', 'pony.txt')
...	...	@@ -187,7 +187,7 @@ def test_host_files(cli, tmpdir, datafiles, path):
187	187	# Test that bind mounts defined in project.conf work
188	188	@pytest.mark.parametrize("path", [("/etc"), ("/usr/share/pony")])
189	189	@pytest.mark.datafiles(DATA_DIR)
190		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	190	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
191	191	def test_host_files_expand_environ(cli, tmpdir, datafiles, path):
192	192	project = os.path.join(datafiles.dirname, datafiles.basename)
193	193	hostpath = os.path.join(project, 'files', 'shell-mount')
...	...	@@ -233,7 +233,7 @@ def test_isolated_no_mount(cli, tmpdir, datafiles, path):
233	233	# declared as optional, and that there is no warning if it is optional
234	234	@pytest.mark.parametrize("optional", [("mandatory"), ("optional")])
235	235	@pytest.mark.datafiles(DATA_DIR)
236		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	236	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
237	237	def test_host_files_missing(cli, tmpdir, datafiles, optional):
238	238	project = os.path.join(datafiles.dirname, datafiles.basename)
239	239	ponyfile = os.path.join(project, 'files', 'shell-mount', 'horsy.txt')
...	...	@@ -269,7 +269,7 @@ def test_host_files_missing(cli, tmpdir, datafiles, optional):
269	269	# Test that bind mounts defined in project.conf work
270	270	@pytest.mark.parametrize("path", [("/etc/pony.conf"), ("/usr/share/pony/pony.txt")])
271	271	@pytest.mark.datafiles(DATA_DIR)
272		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	272	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
273	273	def test_cli_mount(cli, tmpdir, datafiles, path):
274	274	project = os.path.join(datafiles.dirname, datafiles.basename)
275	275	ponyfile = os.path.join(project, 'files', 'shell-mount', 'pony.txt')
...	...	@@ -282,7 +282,7 @@ def test_cli_mount(cli, tmpdir, datafiles, path):
282	282	# Test that we can see the workspace files in a shell
283	283	@pytest.mark.integration
284	284	@pytest.mark.datafiles(DATA_DIR)
285		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	285	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
286	286	def test_workspace_visible(cli, tmpdir, datafiles):
287	287	project = os.path.join(datafiles.dirname, datafiles.basename)
288	288	workspace = os.path.join(cli.directory, 'workspace')
...	...	@@ -316,7 +316,7 @@ def test_workspace_visible(cli, tmpdir, datafiles):
316	316
317	317	# Test that '--sysroot' works
318	318	@pytest.mark.datafiles(DATA_DIR)
319		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	319	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
320	320	def test_sysroot(cli, tmpdir, datafiles):
321	321	project = os.path.join(datafiles.dirname, datafiles.basename)
322	322	base_element = "base/base-alpine.bst"
...	...	@@ -346,7 +346,7 @@ def test_sysroot(cli, tmpdir, datafiles):
346	346
347	347	# Test system integration commands can access devices in /dev
348	348	@pytest.mark.datafiles(DATA_DIR)
349		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	349	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
350	350	def test_integration_devices(cli, tmpdir, datafiles):
351	351	project = os.path.join(datafiles.dirname, datafiles.basename)
352	352	element_name = 'integration.bst'
...	...	@@ -359,7 +359,7 @@ def test_integration_devices(cli, tmpdir, datafiles):
359	359	@pytest.mark.datafiles(DATA_DIR)
360	360	@pytest.mark.parametrize("build_shell", [("build"), ("nobuild")])
361	361	@pytest.mark.parametrize("guess_element", [True, False], ids=["guess", "no-guess"])
362		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	362	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
363	363	def test_integration_external_workspace(cli, tmpdir_factory, datafiles, build_shell, guess_element):
364	364	tmpdir = tmpdir_factory.mktemp("")
365	365	project = os.path.join(datafiles.dirname, datafiles.basename)

tests/integration/sockets.py

---

...	...	@@ -5,7 +5,7 @@ from buildstream import _yaml
5	5
6	6	from tests.testutils import cli_integration as cli
7	7	from tests.testutils.integration import assert_contains
8		-from tests.testutils.site import HAVE_BWRAP, IS_LINUX
	8	+from tests.testutils.site import HAVE_SANDBOX
9	9
10	10
11	11	pytestmark = pytest.mark.integration
...	...	@@ -17,7 +17,7 @@ DATA_DIR = os.path.join(
17	17
18	18
19	19	@pytest.mark.datafiles(DATA_DIR)
20		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	20	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
21	21	def test_builddir_socket_ignored(cli, tmpdir, datafiles):
22	22	project = os.path.join(datafiles.dirname, datafiles.basename)
23	23	element_name = 'sockets/make-builddir-socket.bst'
...	...	@@ -27,7 +27,7 @@ def test_builddir_socket_ignored(cli, tmpdir, datafiles):
27	27
28	28
29	29	@pytest.mark.datafiles(DATA_DIR)
30		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	30	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
31	31	def test_install_root_socket_ignored(cli, tmpdir, datafiles):
32	32	project = os.path.join(datafiles.dirname, datafiles.basename)
33	33	element_name = 'sockets/make-install-root-socket.bst'

tests/integration/source-determinism.py

---

...	...	@@ -4,7 +4,7 @@ import pytest
4	4	from buildstream import _yaml, utils
5	5	from tests.testutils import create_repo, ALL_REPO_KINDS
6	6	from tests.testutils import cli_integration as cli
7		-from tests.testutils.site import HAVE_BWRAP, IS_LINUX
	7	+from tests.testutils.site import HAVE_SANDBOX
8	8
9	9
10	10	DATA_DIR = os.path.join(
...	...	@@ -30,7 +30,7 @@ def create_test_directory(*path, mode=0o644):
30	30	@pytest.mark.integration
31	31	@pytest.mark.datafiles(DATA_DIR)
32	32	@pytest.mark.parametrize("kind", [(kind) for kind in ALL_REPO_KINDS] + ['local'])
33		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	33	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
34	34	def test_deterministic_source_umask(cli, tmpdir, datafiles, kind, integration_cache):
35	35	project = str(datafiles)
36	36	element_name = 'list.bst'
...	...	@@ -103,7 +103,7 @@ def test_deterministic_source_umask(cli, tmpdir, datafiles, kind, integration_ca
103	103
104	104	@pytest.mark.integration
105	105	@pytest.mark.datafiles(DATA_DIR)
106		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	106	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
107	107	def test_deterministic_source_local(cli, tmpdir, datafiles, integration_cache):
108	108	"""Only user rights should be considered for local source.
109	109	"""

tests/integration/stack.py

---

...	...	@@ -4,7 +4,7 @@ import pytest
4	4	from buildstream import _yaml
5	5
6	6	from tests.testutils import cli_integration as cli
7		-from tests.testutils.site import HAVE_BWRAP, IS_LINUX
	7	+from tests.testutils.site import HAVE_SANDBOX
8	8
9	9
10	10	pytestmark = pytest.mark.integration
...	...	@@ -17,7 +17,7 @@ DATA_DIR = os.path.join(
17	17
18	18
19	19	@pytest.mark.datafiles(DATA_DIR)
20		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	20	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
21	21	def test_stack(cli, tmpdir, datafiles):
22	22	project = os.path.join(datafiles.dirname, datafiles.basename)
23	23	checkout = os.path.join(cli.directory, 'checkout')

tests/integration/symlinks.py

---

...	...	@@ -6,7 +6,7 @@ from buildstream import _yaml
6	6
7	7	from tests.testutils import cli_integration as cli
8	8	from tests.testutils.integration import assert_contains
9		-from tests.testutils.site import HAVE_BWRAP, IS_LINUX
	9	+from tests.testutils.site import HAVE_SANDBOX
10	10
11	11
12	12	pytestmark = pytest.mark.integration
...	...	@@ -19,7 +19,7 @@ DATA_DIR = os.path.join(
19	19
20	20
21	21	@pytest.mark.datafiles(DATA_DIR)
22		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	22	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
23	23	def test_absolute_symlinks_made_relative(cli, tmpdir, datafiles):
24	24	project = os.path.join(datafiles.dirname, datafiles.basename)
25	25	checkout = os.path.join(cli.directory, 'checkout')
...	...	@@ -43,7 +43,7 @@ def test_absolute_symlinks_made_relative(cli, tmpdir, datafiles):
43	43
44	44
45	45	@pytest.mark.datafiles(DATA_DIR)
46		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	46	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
47	47	def test_allow_overlaps_inside_symlink_with_dangling_target(cli, tmpdir, datafiles):
48	48	project = os.path.join(datafiles.dirname, datafiles.basename)
49	49	checkout = os.path.join(cli.directory, 'checkout')
...	...	@@ -60,7 +60,7 @@ def test_allow_overlaps_inside_symlink_with_dangling_target(cli, tmpdir, datafil
60	60
61	61
62	62	@pytest.mark.datafiles(DATA_DIR)
63		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	63	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
64	64	def test_detect_symlink_overlaps_pointing_outside_sandbox(cli, tmpdir, datafiles):
65	65	project = os.path.join(datafiles.dirname, datafiles.basename)
66	66	checkout = os.path.join(cli.directory, 'checkout')

tests/integration/workspace.py

---

...	...	@@ -3,7 +3,7 @@ import pytest
3	3
4	4	from buildstream import _yaml
5	5	from tests.testutils import cli_integration as cli
6		-from tests.testutils.site import HAVE_BWRAP, IS_LINUX
	6	+from tests.testutils.site import HAVE_SANDBOX
7	7	from tests.testutils.integration import walk_dir
8	8
9	9
...	...	@@ -18,7 +18,7 @@ DATA_DIR = os.path.join(
18	18
19	19	@pytest.mark.integration
20	20	@pytest.mark.datafiles(DATA_DIR)
21		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	21	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
22	22	def test_workspace_mount(cli, tmpdir, datafiles):
23	23	project = os.path.join(datafiles.dirname, datafiles.basename)
24	24	workspace = os.path.join(cli.directory, 'workspace')
...	...	@@ -35,7 +35,7 @@ def test_workspace_mount(cli, tmpdir, datafiles):
35	35
36	36	@pytest.mark.integration
37	37	@pytest.mark.datafiles(DATA_DIR)
38		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	38	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
39	39	def test_workspace_commanddir(cli, tmpdir, datafiles):
40	40	project = os.path.join(datafiles.dirname, datafiles.basename)
41	41	workspace = os.path.join(cli.directory, 'workspace')
...	...	@@ -53,7 +53,7 @@ def test_workspace_commanddir(cli, tmpdir, datafiles):
53	53
54	54	@pytest.mark.integration
55	55	@pytest.mark.datafiles(DATA_DIR)
56		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	56	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
57	57	def test_workspace_updated_dependency(cli, tmpdir, datafiles):
58	58	project = os.path.join(datafiles.dirname, datafiles.basename)
59	59	workspace = os.path.join(cli.directory, 'workspace')
...	...	@@ -108,7 +108,7 @@ def test_workspace_updated_dependency(cli, tmpdir, datafiles):
108	108
109	109	@pytest.mark.integration
110	110	@pytest.mark.datafiles(DATA_DIR)
111		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	111	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
112	112	def test_workspace_update_dependency_failed(cli, tmpdir, datafiles):
113	113	project = os.path.join(datafiles.dirname, datafiles.basename)
114	114	workspace = os.path.join(cli.directory, 'workspace')
...	...	@@ -184,7 +184,7 @@ def test_workspace_update_dependency_failed(cli, tmpdir, datafiles):
184	184
185	185	@pytest.mark.integration
186	186	@pytest.mark.datafiles(DATA_DIR)
187		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	187	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
188	188	def test_updated_dependency_nested(cli, tmpdir, datafiles):
189	189	project = os.path.join(datafiles.dirname, datafiles.basename)
190	190	workspace = os.path.join(cli.directory, 'workspace')
...	...	@@ -238,7 +238,7 @@ def test_updated_dependency_nested(cli, tmpdir, datafiles):
238	238
239	239	@pytest.mark.integration
240	240	@pytest.mark.datafiles(DATA_DIR)
241		-@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	241	+@pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
242	242	def test_incremental_configure_commands_run_only_once(cli, tmpdir, datafiles):
243	243	project = os.path.join(datafiles.dirname, datafiles.basename)
244	244	workspace = os.path.join(cli.directory, 'workspace')

tests/testutils/filetypegenerator.py

---

...	...	@@ -56,7 +56,14 @@ def generate_file_types(path):
56	56	yield
57	57	clean()
58	58
	59	+ # Change directory because the full path may be longer than the ~100
	60	+ # characters permitted for a unix socket
	61	+ old_dir = os.getcwd()
	62	+ parent, child = os.path.split(path)
	63	+ os.chdir(parent)
	64	+
59	65	s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
60		- s.bind(path)
	66	+ s.bind(child)
	67	+ os.chdir(old_dir)
61	68	yield
62	69	clean()

tests/testutils/site.py

---

...	...	@@ -4,6 +4,7 @@
4	4	import os
5	5	import subprocess
6	6	import sys
	7	+import platform
7	8
8	9	from buildstream import _site, utils, ProgramNotFoundError
9	10	from buildstream._platform import Platform
...	...	@@ -57,6 +58,16 @@ except ImportError:
57	58	HAVE_ARPY = False
58	59
59	60	IS_LINUX = os.getenv('BST_FORCE_BACKEND', sys.platform).startswith('linux')
	61	+IS_WSL = (IS_LINUX and 'Microsoft' in platform.uname().release)
60	62	IS_WINDOWS = (os.name == 'nt')
61	63
	64	+if not IS_LINUX:
	65	+ HAVE_SANDBOX = True # fallback to a chroot sandbox on unix
	66	+elif IS_WSL:
	67	+ HAVE_SANDBOX = False # Sandboxes are inoperable under WSL due to lack of FUSE
	68	+elif IS_LINUX and HAVE_BWRAP:
	69	+ HAVE_SANDBOX = True
	70	+else:
	71	+ HAVE_SANDBOX = False
	72	+
62	73	MACHINE_ARCH = Platform.get_host_arch()

—
View it on GitLab.
You're receiving this email because of your account on gitlab.com. If you'd like to receive fewer emails, you can adjust your notification settings.