[Notes] [Git][BuildStream/buildstream][willsalmon/defaultWorkspaces] 5 commits: contributing: add guidance on unit tests

[Will Salmon <gitlab mg gitlab com>]

Title: GitLab

Will Salmon pushed to branch willsalmon/defaultWorkspaces at BuildStream / buildstream

Commits:

• 082b3811
  by Angelos Evripiotis at 2018-11-15T19:48:12Z

  contributing: add guidance on unit tests
  
  Decrease uncertainty around whether unit tests are welcome in the
  project or not.
  

• b5b79056
  by Jürg Billeter at 2018-11-15T20:24:58Z

  Merge branch 'aevri/unit_tests' into 'master'
  
  contributing: add guidance on unit tests
  
  See merge request BuildStream/buildstream!943

• b0414d49
  by William Salmon at 2018-11-16T10:24:10Z

  tests/frontend/workspace.py: Refactor support function
  
  The Refactor now makes it possible to test for the forth coming multi
  element workspace open commands and to test for failure.
  

• 1cd06080
  by William Salmon at 2018-11-16T10:24:10Z

  Workspace CLI update
  
  This is to update the workspace CLI to as agreed on the mailing list
  https://mail.gnome.org/archives/buildstream-list/2018-September/msg00046.html
  
  This patch also introduces the default workspace directory.
  

• 0f7e4cf2
  by William Salmon at 2018-11-16T10:24:10Z

  Added a news item for workspace open changes
  

15 changed files:

• CONTRIBUTING.rst
• NEWS
• buildstream/_context.py
• buildstream/_frontend/cli.py
• buildstream/_stream.py
• buildstream/data/userconfig.yaml
• doc/sessions/developing.run
• tests/examples/developing.py
• tests/examples/junctions.py
• tests/frontend/buildcheckout.py
• tests/frontend/cross_junction_workspace.py
• tests/frontend/workspace.py
• tests/integration/shell.py
• tests/integration/workspace.py
• tests/plugins/filter.py

Changes:

CONTRIBUTING.rst

---

...	...	@@ -1547,6 +1547,24 @@ Tests that run a sandbox should be decorated with::
1547	1547
1548	1548	and use the integration cli helper.
1549	1549
	1550	+You should first aim to write tests that exercise your changes from the cli.
	1551	+This is so that the testing is end-to-end, and the changes are guaranteed to
	1552	+work for the end-user. The cli is considered stable, and so tests written in
	1553	+terms of it are unlikely to require updating as the internals of the software
	1554	+change over time.
	1555	+
	1556	+It may be impractical to sufficiently examine some changes this way. For
	1557	+example, the number of cases to test and the running time of each test may be
	1558	+too high. It may also be difficult to contrive circumstances to cover every
	1559	+line of the change. If this is the case, next you can consider also writing
	1560	+unit tests that work more directly on the changes.
	1561	+
	1562	+It is important to write unit tests in such a way that they do not break due to
	1563	+changes unrelated to what they are meant to test. For example, if the test
	1564	+relies on a lot of BuildStream internals, a large refactoring will likely
	1565	+require the test to be rewritten. Pure functions that only rely on the Python
	1566	+Standard Library are excellent candidates for unit testing.
	1567	+
1550	1568
1551	1569	Measuring performance
1552	1570	---------------------

NEWS

---

...	...	@@ -45,6 +45,10 @@ buildstream 1.3.1
45	45	instead of just a specially-formatted build-root with a `root` and `scratch`
46	46	subdirectory.
47	47
	48	+ o `bst workspace open` now supports the creation of multiple elements and
	49	+ allows the user to set a default location for there creation. This has meant
	50	+ that the new CLI is no longer backwards compatible with buildstream 1.2.
	51	+
48	52
49	53	=================
50	54	buildstream 1.1.5

buildstream/_context.py

---

...	...	@@ -59,6 +59,9 @@ class Context():
59	59	# The directory where build sandboxes will be created
60	60	self.builddir = None
61	61
	62	+ # Default root location for workspaces
	63	+ self.workspacedir = None
	64	+
62	65	# The local binary artifact cache directory
63	66	self.artifactdir = None
64	67
...	...	@@ -160,10 +163,10 @@ class Context():
160	163	_yaml.node_validate(defaults, [
161	164	'sourcedir', 'builddir', 'artifactdir', 'logdir',
162	165	'scheduler', 'artifacts', 'logging', 'projects',
163		- 'cache'
	166	+ 'cache', 'workspacedir',
164	167	])
165	168
166		- for directory in ['sourcedir', 'builddir', 'artifactdir', 'logdir']:
	169	+ for directory in ['sourcedir', 'builddir', 'artifactdir', 'logdir', 'workspacedir']:
167	170	# Allow the ~ tilde expansion and any environment variables in
168	171	# path specification in the config files.
169	172	#

buildstream/_frontend/cli.py

---

...	...	@@ -678,31 +678,35 @@ def workspace():
678	678	@click.option('--no-checkout', default=False, is_flag=True,
679	679	help="Do not checkout the source, only link to the given directory")
680	680	@click.option('--force', '-f', default=False, is_flag=True,
681		- help="Overwrite files existing in checkout directory")
	681	+ help="The workspace will be created even if the directory in which it will be created is not empty " +
	682	+ "or if a workspace for that element already exists")
682	683	@click.option('--track', 'track_', default=False, is_flag=True,
683	684	help="Track and fetch new source references before checking out the workspace")
684		-@click.argument('element',
685		- type=click.Path(readable=False))
686		-@click.argument('directory', type=click.Path(file_okay=False))
	685	+@click.option('--directory', type=click.Path(file_okay=False), default=None,
	686	+ help="Only for use when a single Element is given: Set the directory to use to create the workspace")
	687	+@click.argument('elements', nargs=-1, type=click.Path(readable=False))
687	688	@click.pass_obj
688		-def workspace_open(app, no_checkout, force, track_, element, directory):
	689	+def workspace_open(app, no_checkout, force, track_, directory, elements):
689	690	"""Open a workspace for manual source modification"""
690		-
691		- if os.path.exists(directory):
692		-
693		- if not os.path.isdir(directory):
694		- click.echo("Checkout directory is not a directory: {}".format(directory), err=True)
	691	+ if directory is not None:
	692	+ if len(elements) > 1:
	693	+ click.echo("Directory option can only be used if a single element is given", err=True)
695	694	sys.exit(-1)
	695	+ if os.path.exists(directory):
	696	+ if not os.path.isdir(directory):
	697	+ click.echo("Directory path is not a directory: {}".format(directory), err=True)
	698	+ sys.exit(-1)
696	699
697		- if not (no_checkout or force) and os.listdir(directory):
698		- click.echo("Checkout directory is not empty: {}".format(directory), err=True)
699		- sys.exit(-1)
	700	+ if not (no_checkout or force) and os.listdir(directory):
	701	+ click.echo("Directory path is not empty: {}".format(directory), err=True)
	702	+ sys.exit(-1)
700	703
701	704	with app.initialized():
702		- app.stream.workspace_open(element, directory,
	705	+ app.stream.workspace_open(elements,
703	706	no_checkout=no_checkout,
704	707	track_first=track_,
705		- force=force)
	708	+ force=force,
	709	+ custom_dir=directory)
706	710
707	711
708	712	##################################################################

buildstream/_stream.py

---

...	...	@@ -448,44 +448,29 @@ class Stream():
448	448	# Open a project workspace
449	449	#
450	450	# Args:
451		- # target (str): The target element to open the workspace for
452		- # directory (str): The directory to stage the source in
	451	+ # target (list): List of target elements to open workspaces for
453	452	# no_checkout (bool): Whether to skip checking out the source
454	453	# track_first (bool): Whether to track and fetch first
455	454	# force (bool): Whether to ignore contents in an existing directory
456	455	#
457		- def workspace_open(self, target, directory, *,
	456	+ def workspace_open(self, targets, *,
458	457	no_checkout,
459	458	track_first,
460		- force):
	459	+ force,
	460	+ custom_dir):
	461	+ # This function is a little funny but it is trying to be as atomic as possible.
461	462
462	463	if track_first:
463		- track_targets = (target,)
	464	+ track_targets = targets
464	465	else:
465	466	track_targets = ()
466	467
467		- elements, track_elements = self._load((target,), track_targets,
	468	+ elements, track_elements = self._load(targets, track_targets,
468	469	selection=PipelineSelection.REDIRECT,
469	470	track_selection=PipelineSelection.REDIRECT)
470		- target = elements[0]
471		- directory = os.path.abspath(directory)
472		-
473		- if not list(target.sources()):
474		- build_depends = [x.name for x in target.dependencies(Scope.BUILD, recurse=False)]
475		- if not build_depends:
476		- raise StreamError("The given element has no sources")
477		- detail = "Try opening a workspace on one of its dependencies instead:\n"
478		- detail += " \n".join(build_depends)
479		- raise StreamError("The given element has no sources", detail=detail)
480	471
481	472	workspaces = self._context.get_workspaces()
482	473
483		- # Check for workspace config
484		- workspace = workspaces.get_workspace(target._get_full_name())
485		- if workspace and not force:
486		- raise StreamError("Workspace '{}' is already defined at: {}"
487		- .format(target.name, workspace.get_absolute_path()))
488		-
489	474	# If we're going to checkout, we need at least a fetch,
490	475	# if we were asked to track first, we're going to fetch anyway.
491	476	#
...	...	@@ -495,29 +480,78 @@ class Stream():
495	480	track_elements = elements
496	481	self._fetch(elements, track_elements=track_elements)
497	482
498		- if not no_checkout and target._get_consistency() != Consistency.CACHED:
499		- raise StreamError("Could not stage uncached source. " +
500		- "Use `--track` to track and " +
501		- "fetch the latest version of the " +
502		- "source.")
503		-
504		- if workspace:
505		- workspaces.delete_workspace(target._get_full_name())
506		- workspaces.save_config()
507		- shutil.rmtree(directory)
508		- try:
509		- os.makedirs(directory, exist_ok=True)
510		- except OSError as e:
511		- raise StreamError("Failed to create workspace directory: {}".format(e)) from e
	483	+ expanded_directories = []
	484	+ # To try to be more atomic, loop through the elements and raise any errors we can early
	485	+ for target in elements:
	486	+
	487	+ if not list(target.sources()):
	488	+ build_depends = [x.name for x in target.dependencies(Scope.BUILD, recurse=False)]
	489	+ if not build_depends:
	490	+ raise StreamError("The element {} has no sources".format(target.name))
	491	+ detail = "Try opening a workspace on one of its dependencies instead:\n"
	492	+ detail += " \n".join(build_depends)
	493	+ raise StreamError("The element {} has no sources".format(target.name), detail=detail)
	494	+
	495	+ # Check for workspace config
	496	+ workspace = workspaces.get_workspace(target._get_full_name())
	497	+ if workspace and not force:
	498	+ raise StreamError("Workspace '{}' is already defined at: {}"
	499	+ .format(target.name, workspace.get_absolute_path()))
	500	+
	501	+ if not no_checkout and target._get_consistency() != Consistency.CACHED:
	502	+ raise StreamError("Could not stage uncached source. For {} ".format(target.name) +
	503	+ "Use `--track` to track and " +
	504	+ "fetch the latest version of the " +
	505	+ "source.")
	506	+
	507	+ if not custom_dir:
	508	+ directory = os.path.abspath(os.path.join(self._context.workspacedir, target.name))
	509	+ if directory[-4:] == '.bst':
	510	+ directory = directory[:-4]
	511	+ expanded_directories.append(directory)
	512	+
	513	+ if custom_dir:
	514	+ assert len(elements) == 1
	515	+ # The CLI function that calls this should have caught len(elements) != 1 so if this fails it is a bug that
	516	+ # can not be tested for via the run bst test mechanism.
	517	+ expanded_directories = [custom_dir, ]
	518	+ else:
	519	+ # If this fails it is a bug in what ever calls this, usually cli.py and so can not be tested for via the
	520	+ # run bst test mechanism.
	521	+ assert len(elements) == len(expanded_directories)
	522	+
	523	+ # So far this function has tried to catch as many issues as possible with out making any changes
	524	+ # Now it dose the bits that can not be made atomic.
	525	+ targetGenerator = zip(elements, expanded_directories)
	526	+ for target, directory in targetGenerator:
	527	+ self._message(MessageType.INFO, "Creating workspace for element {}"
	528	+ .format(target.name))
	529	+
	530	+ workspace = workspaces.get_workspace(target._get_full_name())
	531	+ if workspace:
	532	+ workspaces.delete_workspace(target._get_full_name())
	533	+ workspaces.save_config()
	534	+ shutil.rmtree(directory)
	535	+ try:
	536	+ os.makedirs(directory, exist_ok=True)
	537	+ except OSError as e:
	538	+ todo_elements = " ".join([str(target.name) for target, directory_dict in targetGenerator])
	539	+ if todo_elements:
	540	+ # This output should make creating the remaining workspaces as easy as possible.
	541	+ todo_elements = "\nDid not try to create workspaces for " + todo_elements
	542	+ raise StreamError("Failed to create workspace directory: {}".format(e) + todo_elements) from e
512	543
513		- workspaces.create_workspace(target._get_full_name(), directory)
	544	+ workspaces.create_workspace(target._get_full_name(), directory)
514	545
515		- if not no_checkout:
516		- with target.timed_activity("Staging sources to {}".format(directory)):
517		- target._open_workspace()
	546	+ if not no_checkout:
	547	+ with target.timed_activity("Staging sources to {}".format(directory)):
	548	+ target._open_workspace()
518	549
519		- workspaces.save_config()
520		- self._message(MessageType.INFO, "Saved workspace configuration")
	550	+ # Saving the workspace once it is set up means that if the next workspace fails to be created before
	551	+ # the configuration gets saved. The successfully created workspace still gets saved.
	552	+ workspaces.save_config()
	553	+ self._message(MessageType.INFO, "Created a workspace for element: {}"
	554	+ .format(target._get_full_name()))
521	555
522	556	# workspace_close
523	557	#

buildstream/data/userconfig.yaml

---

...	...	@@ -22,6 +22,9 @@ artifactdir: ${XDG_CACHE_HOME}/buildstream/artifacts
22	22	# Location to store build logs
23	23	logdir: ${XDG_CACHE_HOME}/buildstream/logs
24	24
	25	+# Default root location for workspaces, blank for no default set.
	26	+workspacedir: .
	27	+
25	28	#
26	29	# Cache
27	30	#

doc/sessions/developing.run

---

...	...	@@ -7,7 +7,7 @@ commands:
7	7	# Capture workspace open output
8	8	- directory: ../examples/developing/
9	9	output: ../source/sessions/developing-workspace-open.html
10		- command: workspace open hello.bst workspace_hello
	10	+ command: workspace open hello.bst --directory workspace_hello
11	11
12	12	# Catpure output from workspace list
13	13	- directory: ../examples/developing/

tests/examples/developing.py

---

...	...	@@ -59,7 +59,7 @@ def test_open_workspace(cli, tmpdir, datafiles):
59	59	project = os.path.join(datafiles.dirname, datafiles.basename)
60	60	workspace_dir = os.path.join(str(tmpdir), "workspace_hello")
61	61
62		- result = cli.run(project=project, args=['workspace', 'open', '-f', 'hello.bst', workspace_dir])
	62	+ result = cli.run(project=project, args=['workspace', 'open', '-f', '--directory', workspace_dir, 'hello.bst', ])
63	63	result.assert_success()
64	64
65	65	result = cli.run(project=project, args=['workspace', 'list'])
...	...	@@ -78,7 +78,7 @@ def test_make_change_in_workspace(cli, tmpdir, datafiles):
78	78	project = os.path.join(datafiles.dirname, datafiles.basename)
79	79	workspace_dir = os.path.join(str(tmpdir), "workspace_hello")
80	80
81		- result = cli.run(project=project, args=['workspace', 'open', '-f', 'hello.bst', workspace_dir])
	81	+ result = cli.run(project=project, args=['workspace', 'open', '-f', '--directory', workspace_dir, 'hello.bst'])
82	82	result.assert_success()
83	83
84	84	result = cli.run(project=project, args=['workspace', 'list'])

tests/examples/junctions.py

---

...	...	@@ -48,7 +48,7 @@ def test_open_cross_junction_workspace(cli, tmpdir, datafiles):
48	48	workspace_dir = os.path.join(str(tmpdir), "workspace_hello_junction")
49	49
50	50	result = cli.run(project=project,
51		- args=['workspace', 'open', 'hello-junction.bst:hello.bst', workspace_dir])
	51	+ args=['workspace', 'open', '--directory', workspace_dir, 'hello-junction.bst:hello.bst'])
52	52	result.assert_success()
53	53
54	54	result = cli.run(project=project,

tests/frontend/buildcheckout.py

---

...	...	@@ -509,7 +509,7 @@ def test_build_checkout_workspaced_junction(cli, tmpdir, datafiles):
509	509
510	510	# Now open a workspace on the junction
511	511	#
512		- result = cli.run(project=project, args=['workspace', 'open', 'junction.bst', workspace])
	512	+ result = cli.run(project=project, args=['workspace', 'open', '--directory', workspace, 'junction.bst'])
513	513	result.assert_success()
514	514	filename = os.path.join(workspace, 'files', 'etc-files', 'etc', 'animal.conf')
515	515

tests/frontend/cross_junction_workspace.py

---

...	...	@@ -47,7 +47,7 @@ def open_cross_junction(cli, tmpdir):
47	47	workspace = tmpdir.join("workspace")
48	48
49	49	element = 'sub.bst:data.bst'
50		- args = ['workspace', 'open', element, str(workspace)]
	50	+ args = ['workspace', 'open', '--directory', str(workspace), element]
51	51	result = cli.run(project=project, args=args)
52	52	result.assert_success()
53	53

tests/frontend/workspace.py

---

...	...	@@ -21,9 +21,11 @@
21	21	# Phillip Smyth <phillip smyth codethink co uk>
22	22	# Jonathan Maw <jonathan maw codethink co uk>
23	23	# Richard Maw <richard maw codethink co uk>
	24	+# William Salmon <will salmon codethink co uk>
24	25	#
25	26
26	27	import os
	28	+import stat
27	29	import pytest
28	30	import shutil
29	31	import subprocess
...	...	@@ -43,65 +45,120 @@ DATA_DIR = os.path.join(
43	45	)
44	46
45	47
46		-def open_workspace(cli, tmpdir, datafiles, kind, track, suffix='', workspace_dir=None,
47		- project_path=None, element_attrs=None):
48		- if not workspace_dir:
49		- workspace_dir = os.path.join(str(tmpdir), 'workspace{}'.format(suffix))
50		- if not project_path:
51		- project_path = os.path.join(datafiles.dirname, datafiles.basename)
52		- else:
53		- shutil.copytree(os.path.join(datafiles.dirname, datafiles.basename), project_path)
54		- bin_files_path = os.path.join(project_path, 'files', 'bin-files')
55		- element_path = os.path.join(project_path, 'elements')
56		- element_name = 'workspace-test-{}{}.bst'.format(kind, suffix)
	48	+class WorkspaceCreater():
	49	+ def __init__(self, cli, tmpdir, datafiles, project_path=None):
	50	+ self.cli = cli
	51	+ self.tmpdir = tmpdir
	52	+ self.datafiles = datafiles
	53	+
	54	+ if not project_path:
	55	+ project_path = os.path.join(datafiles.dirname, datafiles.basename)
	56	+ else:
	57	+ shutil.copytree(os.path.join(datafiles.dirname, datafiles.basename), project_path)
	58	+
	59	+ self.project_path = project_path
	60	+ self.bin_files_path = os.path.join(project_path, 'files', 'bin-files')
	61	+
	62	+ self.workspace_cmd = os.path.join(self.project_path, 'workspace_cmd')
	63	+
	64	+ def create_workspace_element(self, kind, track, suffix='', workspace_dir=None,
	65	+ element_attrs=None):
	66	+ element_name = 'workspace-test-{}{}.bst'.format(kind, suffix)
	67	+ element_path = os.path.join(self.project_path, 'elements')
	68	+ if not workspace_dir:
	69	+ workspace_dir = os.path.join(self.workspace_cmd, element_name)
	70	+ if workspace_dir[-4:] == '.bst':
	71	+ workspace_dir = workspace_dir[:-4]
	72	+
	73	+ # Create our repo object of the given source type with
	74	+ # the bin files, and then collect the initial ref.
	75	+ repo = create_repo(kind, str(self.tmpdir))
	76	+ ref = repo.create(self.bin_files_path)
	77	+ if track:
	78	+ ref = None
	79	+
	80	+ # Write out our test target
	81	+ element = {
	82	+ 'kind': 'import',
	83	+ 'sources': [
	84	+ repo.source_config(ref=ref)
	85	+ ]
	86	+ }
	87	+ if element_attrs:
	88	+ element = {**element, **element_attrs}
	89	+ _yaml.dump(element,
	90	+ os.path.join(element_path,
	91	+ element_name))
	92	+ return element_name, element_path, workspace_dir
57	93
58		- # Create our repo object of the given source type with
59		- # the bin files, and then collect the initial ref.
60		- #
61		- repo = create_repo(kind, str(tmpdir))
62		- ref = repo.create(bin_files_path)
63		- if track:
64		- ref = None
	94	+ def create_workspace_elements(self, kinds, track, suffixs=None, workspace_dir_usr=None,
	95	+ element_attrs=None):
65	96
66		- # Write out our test target
67		- element = {
68		- 'kind': 'import',
69		- 'sources': [
70		- repo.source_config(ref=ref)
71		- ]
72		- }
73		- if element_attrs:
74		- element = {**element, **element_attrs}
75		- _yaml.dump(element,
76		- os.path.join(element_path,
77		- element_name))
	97	+ element_tuples = []
78	98
79		- # Assert that there is no reference, a track & fetch is needed
80		- state = cli.get_element_state(project_path, element_name)
81		- if track:
82		- assert state == 'no reference'
83		- else:
84		- assert state == 'fetch needed'
	99	+ if suffixs is None:
	100	+ suffixs = ['', ] * len(kinds)
	101	+ else:
	102	+ if len(suffixs) != len(kinds):
	103	+ raise "terable error"
85	104
86		- # Now open the workspace, this should have the effect of automatically
87		- # tracking & fetching the source from the repo.
88		- args = ['workspace', 'open']
89		- if track:
90		- args.append('--track')
91		- args.extend([element_name, workspace_dir])
92		- result = cli.run(project=project_path, args=args)
	105	+ for suffix, kind in zip(suffixs, kinds):
	106	+ element_name, element_path, workspace_dir = \
	107	+ self.create_workspace_element(kind, track, suffix, workspace_dir_usr,
	108	+ element_attrs)
93	109
94		- result.assert_success()
	110	+ # Assert that there is no reference, a track & fetch is needed
	111	+ state = self.cli.get_element_state(self.project_path, element_name)
	112	+ if track:
	113	+ assert state == 'no reference'
	114	+ else:
	115	+ assert state == 'fetch needed'
	116	+ element_tuples.append((element_name, workspace_dir))
95	117
96		- # Assert that we are now buildable because the source is
97		- # now cached.
98		- assert cli.get_element_state(project_path, element_name) == 'buildable'
	118	+ return element_tuples
99	119
100		- # Check that the executable hello file is found in the workspace
101		- filename = os.path.join(workspace_dir, 'usr', 'bin', 'hello')
102		- assert os.path.exists(filename)
	120	+ def open_workspaces(self, kinds, track, suffixs=None, workspace_dir=None,
	121	+ element_attrs=None):
	122	+
	123	+ element_tuples = self.create_workspace_elements(kinds, track, suffixs, workspace_dir,
	124	+ element_attrs)
	125	+ os.makedirs(self.workspace_cmd, exist_ok=True)
	126	+
	127	+ # Now open the workspace, this should have the effect of automatically
	128	+ # tracking & fetching the source from the repo.
	129	+ args = ['workspace', 'open']
	130	+ if track:
	131	+ args.append('--track')
	132	+ if workspace_dir is not None:
	133	+ assert len(element_tuples) == 1, "test logic error"
	134	+ _, workspace_dir = element_tuples[0]
	135	+ args.extend(['--directory', workspace_dir])
	136	+
	137	+ args.extend([element_name for element_name, workspace_dir_suffix in element_tuples])
	138	+ result = self.cli.run(cwd=self.workspace_cmd, project=self.project_path, args=args)
103	139
104		- return (element_name, project_path, workspace_dir)
	140	+ result.assert_success()
	141	+
	142	+ for element_name, workspace_dir in element_tuples:
	143	+ # Assert that we are now buildable because the source is
	144	+ # now cached.
	145	+ assert self.cli.get_element_state(self.project_path, element_name) == 'buildable'
	146	+
	147	+ # Check that the executable hello file is found in the workspace
	148	+ filename = os.path.join(workspace_dir, 'usr', 'bin', 'hello')
	149	+ assert os.path.exists(filename)
	150	+
	151	+ return element_tuples
	152	+
	153	+
	154	+def open_workspace(cli, tmpdir, datafiles, kind, track, suffix='', workspace_dir=None,
	155	+ project_path=None, element_attrs=None):
	156	+ workspace_object = WorkspaceCreater(cli, tmpdir, datafiles, project_path)
	157	+ workspaces = workspace_object.open_workspaces((kind, ), track, (suffix, ), workspace_dir,
	158	+ element_attrs)
	159	+ assert len(workspaces) == 1
	160	+ element_name, workspace = workspaces[0]
	161	+ return element_name, workspace_object.project_path, workspace
105	162
106	163
107	164	@pytest.mark.datafiles(DATA_DIR)
...	...	@@ -128,6 +185,103 @@ def test_open_bzr_customize(cli, tmpdir, datafiles):
128	185	assert(expected_output_str in str(output))
129	186
130	187
	188	+@pytest.mark.datafiles(DATA_DIR)
	189	+def test_open_multi(cli, tmpdir, datafiles):
	190	+
	191	+ workspace_object = WorkspaceCreater(cli, tmpdir, datafiles)
	192	+ workspaces = workspace_object.open_workspaces(repo_kinds, False)
	193	+
	194	+ for (elname, workspace), kind in zip(workspaces, repo_kinds):
	195	+ assert kind in elname
	196	+ workspace_lsdir = os.listdir(workspace)
	197	+ if kind == 'git':
	198	+ assert('.git' in workspace_lsdir)
	199	+ elif kind == 'bzr':
	200	+ assert('.bzr' in workspace_lsdir)
	201	+ else:
	202	+ assert not ('.git' in workspace_lsdir)
	203	+ assert not ('.bzr' in workspace_lsdir)
	204	+
	205	+
	206	+@pytest.mark.datafiles(DATA_DIR)
	207	+def test_open_multi_unwritable(cli, tmpdir, datafiles):
	208	+ workspace_object = WorkspaceCreater(cli, tmpdir, datafiles)
	209	+
	210	+ element_tuples = workspace_object.create_workspace_elements(repo_kinds, False, repo_kinds)
	211	+ os.makedirs(workspace_object.workspace_cmd, exist_ok=True)
	212	+
	213	+ # Now open the workspace, this should have the effect of automatically
	214	+ # tracking & fetching the source from the repo.
	215	+ args = ['workspace', 'open']
	216	+ args.extend([element_name for element_name, workspace_dir_suffix in element_tuples])
	217	+ cli.configure({'workspacedir': workspace_object.workspace_cmd})
	218	+
	219	+ cwdstat = os.stat(workspace_object.workspace_cmd)
	220	+ try:
	221	+ os.chmod(workspace_object.workspace_cmd, cwdstat.st_mode - stat.S_IWRITE)
	222	+ result = workspace_object.cli.run(project=workspace_object.project_path, args=args)
	223	+ finally:
	224	+ # Using this finally to make sure we always put thing back how they should be.
	225	+ os.chmod(workspace_object.workspace_cmd, cwdstat.st_mode)
	226	+
	227	+ result.assert_main_error(ErrorDomain.STREAM, None)
	228	+ # Normally we avoid checking stderr in favour of using the mechine readable result.assert_main_error
	229	+ # But Tristan was very keen that the names of the elements left needing workspaces were present in the out put
	230	+ assert (" ".join([element_name for element_name, workspace_dir_suffix in element_tuples[1:]]) in result.stderr)
	231	+
	232	+
	233	+@pytest.mark.datafiles(DATA_DIR)
	234	+def test_open_multi_with_directory(cli, tmpdir, datafiles):
	235	+ workspace_object = WorkspaceCreater(cli, tmpdir, datafiles)
	236	+
	237	+ element_tuples = workspace_object.create_workspace_elements(repo_kinds, False, repo_kinds)
	238	+ os.makedirs(workspace_object.workspace_cmd, exist_ok=True)
	239	+
	240	+ # Now open the workspace, this should have the effect of automatically
	241	+ # tracking & fetching the source from the repo.
	242	+ args = ['workspace', 'open']
	243	+ args.extend(['--directory', 'any/dir/should/fail'])
	244	+
	245	+ args.extend([element_name for element_name, workspace_dir_suffix in element_tuples])
	246	+ result = workspace_object.cli.run(cwd=workspace_object.workspace_cmd, project=workspace_object.project_path,
	247	+ args=args)
	248	+
	249	+ result.assert_main_error(ErrorDomain.CAS, None)
	250	+ assert ("Directory option can only be used if a single element is given" in result.stderr)
	251	+
	252	+
	253	+@pytest.mark.datafiles(DATA_DIR)
	254	+def test_open_defaultlocation(cli, tmpdir, datafiles):
	255	+ workspace_object = WorkspaceCreater(cli, tmpdir, datafiles)
	256	+
	257	+ ((element_name, workspace_dir), ) = workspace_object.create_workspace_elements(['git'], False, ['git'])
	258	+ os.makedirs(workspace_object.workspace_cmd, exist_ok=True)
	259	+
	260	+ # Now open the workspace, this should have the effect of automatically
	261	+ # tracking & fetching the source from the repo.
	262	+ args = ['workspace', 'open']
	263	+ args.append(element_name)
	264	+
	265	+ # In the other tests we set the cmd to workspace_object.workspace_cmd with the optional
	266	+ # argument, cwd for the workspace_object.cli.run function. But hear we set the default
	267	+ # workspace location to workspace_object.workspace_cmd and run the cli.run function with
	268	+ # no cwd option so that it runs in the project directory.
	269	+ cli.configure({'workspacedir': workspace_object.workspace_cmd})
	270	+ result = workspace_object.cli.run(project=workspace_object.project_path,
	271	+ args=args)
	272	+
	273	+ result.assert_success()
	274	+
	275	+ assert cli.get_element_state(workspace_object.project_path, element_name) == 'buildable'
	276	+
	277	+ # Check that the executable hello file is found in the workspace
	278	+ # even though the cli.run function was not run with cwd = workspace_object.workspace_cmd
	279	+ # the workspace should be created in there as we used the 'workspacedir' configuration
	280	+ # option.
	281	+ filename = os.path.join(workspace_dir, 'usr', 'bin', 'hello')
	282	+ assert os.path.exists(filename)
	283	+
	284	+
131	285	@pytest.mark.datafiles(DATA_DIR)
132	286	@pytest.mark.parametrize("kind", repo_kinds)
133	287	def test_open_track(cli, tmpdir, datafiles, kind):
...	...	@@ -150,7 +304,7 @@ def test_open_force(cli, tmpdir, datafiles, kind):
150	304
151	305	# Now open the workspace again with --force, this should happily succeed
152	306	result = cli.run(project=project, args=[
153		- 'workspace', 'open', '--force', element_name, workspace
	307	+ 'workspace', 'open', '--force', '--directory', workspace, element_name
154	308	])
155	309	result.assert_success()
156	310
...	...	@@ -165,7 +319,7 @@ def test_open_force_open(cli, tmpdir, datafiles, kind):
165	319
166	320	# Now open the workspace again with --force, this should happily succeed
167	321	result = cli.run(project=project, args=[
168		- 'workspace', 'open', '--force', element_name, workspace
	322	+ 'workspace', 'open', '--force', '--directory', workspace, element_name
169	323	])
170	324	result.assert_success()
171	325
...	...	@@ -196,7 +350,7 @@ def test_open_force_different_workspace(cli, tmpdir, datafiles, kind):
196	350
197	351	# Now open the workspace again with --force, this should happily succeed
198	352	result = cli.run(project=project, args=[
199		- 'workspace', 'open', '--force', element_name2, workspace
	353	+ 'workspace', 'open', '--force', '--directory', workspace, element_name2
200	354	])
201	355
202	356	# Assert that the file in workspace 1 has been replaced
...	...	@@ -504,7 +658,7 @@ def test_buildable_no_ref(cli, tmpdir, datafiles):
504	658	# Now open the workspace. We don't need to checkout the source though.
505	659	workspace = os.path.join(str(tmpdir), 'workspace-no-ref')
506	660	os.makedirs(workspace)
507		- args = ['workspace', 'open', '--no-checkout', element_name, workspace]
	661	+ args = ['workspace', 'open', '--no-checkout', '--directory', workspace, element_name]
508	662	result = cli.run(project=project, args=args)
509	663	result.assert_success()
510	664
...	...	@@ -766,7 +920,7 @@ def test_list_supported_workspace(cli, tmpdir, datafiles, workspace_cfg, expecte
766	920	element_name))
767	921
768	922	# Make a change to the workspaces file
769		- result = cli.run(project=project, args=['workspace', 'open', element_name, workspace])
	923	+ result = cli.run(project=project, args=['workspace', 'open', '--directory', workspace, element_name])
770	924	result.assert_success()
771	925	result = cli.run(project=project, args=['workspace', 'close', '--remove-dir', element_name])
772	926	result.assert_success()

tests/integration/shell.py

---

...	...	@@ -278,7 +278,7 @@ def test_workspace_visible(cli, tmpdir, datafiles):
278	278
279	279	# Open a workspace on our build failing element
280	280	#
281		- res = cli.run(project=project, args=['workspace', 'open', element_name, workspace])
	281	+ res = cli.run(project=project, args=['workspace', 'open', '--directory', workspace, element_name])
282	282	assert res.exit_code == 0
283	283
284	284	# Ensure the dependencies of our build failing element are built

tests/integration/workspace.py

---

...	...	@@ -23,7 +23,7 @@ def test_workspace_mount(cli, tmpdir, datafiles):
23	23	workspace = os.path.join(cli.directory, 'workspace')
24	24	element_name = 'workspace/workspace-mount.bst'
25	25
26		- res = cli.run(project=project, args=['workspace', 'open', element_name, workspace])
	26	+ res = cli.run(project=project, args=['workspace', 'open', '--directory', workspace, element_name])
27	27	assert res.exit_code == 0
28	28
29	29	res = cli.run(project=project, args=['build', element_name])
...	...	@@ -39,7 +39,7 @@ def test_workspace_commanddir(cli, tmpdir, datafiles):
39	39	workspace = os.path.join(cli.directory, 'workspace')
40	40	element_name = 'workspace/workspace-commanddir.bst'
41	41
42		- res = cli.run(project=project, args=['workspace', 'open', element_name, workspace])
	42	+ res = cli.run(project=project, args=['workspace', 'open', '--directory', workspace, element_name])
43	43	assert res.exit_code == 0
44	44
45	45	res = cli.run(project=project, args=['build', element_name])
...	...	@@ -75,7 +75,7 @@ def test_workspace_updated_dependency(cli, tmpdir, datafiles):
75	75	_yaml.dump(dependency, os.path.join(element_path, dep_name))
76	76
77	77	# First open the workspace
78		- res = cli.run(project=project, args=['workspace', 'open', element_name, workspace])
	78	+ res = cli.run(project=project, args=['workspace', 'open', '--directory', workspace, element_name])
79	79	assert res.exit_code == 0
80	80
81	81	# We build the workspaced element, so that we have an artifact
...	...	@@ -130,7 +130,7 @@ def test_workspace_update_dependency_failed(cli, tmpdir, datafiles):
130	130	_yaml.dump(dependency, os.path.join(element_path, dep_name))
131	131
132	132	# First open the workspace
133		- res = cli.run(project=project, args=['workspace', 'open', element_name, workspace])
	133	+ res = cli.run(project=project, args=['workspace', 'open', '--directory', workspace, element_name])
134	134	assert res.exit_code == 0
135	135
136	136	# We build the workspaced element, so that we have an artifact
...	...	@@ -205,7 +205,7 @@ def test_updated_dependency_nested(cli, tmpdir, datafiles):
205	205	_yaml.dump(dependency, os.path.join(element_path, dep_name))
206	206
207	207	# First open the workspace
208		- res = cli.run(project=project, args=['workspace', 'open', element_name, workspace])
	208	+ res = cli.run(project=project, args=['workspace', 'open', '--directory', workspace, element_name])
209	209	assert res.exit_code == 0
210	210
211	211	# We build the workspaced element, so that we have an artifact
...	...	@@ -258,7 +258,7 @@ def test_incremental_configure_commands_run_only_once(cli, tmpdir, datafiles):
258	258	_yaml.dump(element, os.path.join(element_path, element_name))
259	259
260	260	# We open a workspace on the above element
261		- res = cli.run(project=project, args=['workspace', 'open', element_name, workspace])
	261	+ res = cli.run(project=project, args=['workspace', 'open', '--directory', workspace, element_name])
262	262	res.assert_success()
263	263
264	264	# Then we build, and check whether the configure step succeeded

tests/plugins/filter.py

---

...	...	@@ -108,19 +108,25 @@ def test_filter_forbid_also_rdep(datafiles, cli):
108	108	def test_filter_workspace_open(datafiles, cli, tmpdir):
109	109	project = os.path.join(datafiles.dirname, datafiles.basename)
110	110	workspace_dir = os.path.join(tmpdir.dirname, tmpdir.basename, "workspace")
111		- result = cli.run(project=project, args=['workspace', 'open', 'deps-permitted.bst', workspace_dir])
	111	+ result = cli.run(project=project, args=['workspace', 'open', '--directory', workspace_dir, 'deps-permitted.bst'])
112	112	result.assert_success()
113	113	assert os.path.exists(os.path.join(workspace_dir, "foo"))
114	114	assert os.path.exists(os.path.join(workspace_dir, "bar"))
115	115	assert os.path.exists(os.path.join(workspace_dir, "baz"))
116	116
	117	+@pytest.mark.datafiles(os.path.join(DATA_DIR, 'basic'))
	118	+def test_filter_workspace_open_multi(datafiles, cli, tmpdir):
	119	+ project = os.path.join(datafiles.dirname, datafiles.basename)
	120	+ result = cli.run(cwd=project, project=project, args=['workspace', 'open', 'deps-permitted.bst', 'output-orphans.bst'])
	121	+ result.assert_success()
	122	+ assert os.path.exists(os.path.join(project, "input"))
117	123
118	124	@pytest.mark.datafiles(os.path.join(DATA_DIR, 'basic'))
119	125	def test_filter_workspace_build(datafiles, cli, tmpdir):
120	126	project = os.path.join(datafiles.dirname, datafiles.basename)
121	127	tempdir = os.path.join(tmpdir.dirname, tmpdir.basename)
122	128	workspace_dir = os.path.join(tempdir, "workspace")
123		- result = cli.run(project=project, args=['workspace', 'open', 'output-orphans.bst', workspace_dir])
	129	+ result = cli.run(project=project, args=['workspace', 'open', '--directory', workspace_dir, 'output-orphans.bst'])
124	130	result.assert_success()
125	131	src = os.path.join(workspace_dir, "foo")
126	132	dst = os.path.join(workspace_dir, "quux")
...	...	@@ -138,7 +144,7 @@ def test_filter_workspace_close(datafiles, cli, tmpdir):
138	144	project = os.path.join(datafiles.dirname, datafiles.basename)
139	145	tempdir = os.path.join(tmpdir.dirname, tmpdir.basename)
140	146	workspace_dir = os.path.join(tempdir, "workspace")
141		- result = cli.run(project=project, args=['workspace', 'open', 'output-orphans.bst', workspace_dir])
	147	+ result = cli.run(project=project, args=['workspace', 'open', '--directory', workspace_dir, 'output-orphans.bst'])
142	148	result.assert_success()
143	149	src = os.path.join(workspace_dir, "foo")
144	150	dst = os.path.join(workspace_dir, "quux")
...	...	@@ -158,7 +164,7 @@ def test_filter_workspace_reset(datafiles, cli, tmpdir):
158	164	project = os.path.join(datafiles.dirname, datafiles.basename)
159	165	tempdir = os.path.join(tmpdir.dirname, tmpdir.basename)
160	166	workspace_dir = os.path.join(tempdir, "workspace")
161		- result = cli.run(project=project, args=['workspace', 'open', 'output-orphans.bst', workspace_dir])
	167	+ result = cli.run(project=project, args=['workspace', 'open', '--directory', workspace_dir, 'output-orphans.bst'])
162	168	result.assert_success()
163	169	src = os.path.join(workspace_dir, "foo")
164	170	dst = os.path.join(workspace_dir, "quux")

—
View it on GitLab.
You're receiving this email because of your account on gitlab.com. If you'd like to receive fewer emails, you can adjust your notification settings.