[Notes] [Git][BuildStream/buildstream][jonathan/workspace-fragment-guess-element] 10 commits: _project.py: Find project from workspace if outside of a project

[Jonathan Maw <gitlab mg gitlab com>]

Title: GitLab

Jonathan Maw pushed to branch jonathan/workspace-fragment-guess-element at BuildStream / buildstream

Commits:

• b22bc1c2
  by Jonathan Maw at 2018-11-27T17:08:39Z

  _project.py: Find project from workspace if outside of a project
  
  Searching for the workspace file is preferred to searching for the
  project file for the moment, because it avoids the circumstance of
  the workspaces being inside a directory where a higher-level directory
  may have project.conf (but may not be a buildstream project).
  

• 1e3a0c60
  by Jonathan Maw at 2018-11-27T17:08:43Z

  _stream.py: Opened workspaces store the project that opened them
  

• 33a911cc
  by Jonathan Maw at 2018-11-27T17:08:43Z

  _stream.py: workspace_close deletes the stored project reference
  

• c0ebef05
  by Jonathan Maw at 2018-11-27T17:08:43Z

  _stream.py: Re-create the .bstproject when resetting the workspace
  

• dcb13f54
  by Jonathan Maw at 2018-11-27T17:08:43Z

  _workspaces.py: Do not include .bstproject.yaml in the cache key
  

• 25a9c86e
  by Jonathan Maw at 2018-11-27T17:08:43Z

  cli: Interactively warn if the user is trying to close the workspace they're using to load the project
  
  This involves changes in:
  * _stream.py:
    * Add the helper Stream.workspace_is_required()
  * userconfig.yaml:
    * Add a default value for prompt.really-workspace-close-project-inaccessible
  * _context.py:
    * Load the prompt 'really-workspace-close-project-inaccessible' from
      user config.
  * cli.py:
    * If buildstream is invoked interactively, prompt the user to confirm
      that they want to close the workspace they're using to load this
      project.
  

• 1e644e67
  by Jonathan Maw at 2018-11-27T17:08:44Z

  tests: Test bst commands from an external workspace
  

• e4797e80
  by Jonathan Maw at 2018-11-27T17:08:44Z

  NEWS: Add an entry for being able to run commands from a workspace
  

• e6bce3d0
  by Jonathan Maw at 2018-11-27T17:09:06Z

  WIP: Make specifying elements optional in bst commands
  
  Known issues:
  * `bst shell` works, but `bst shell COMMANDS...` doesn't, because click
    has no way of separating optional args from variable-length args.
  * `bst checkout`'s usage string marks LOCATION as an optional argument.
    Because click gets confused if there's an optional argument before a
    mandatory argument, I had to mark LOCATION as optional internally.
  * `bst workspace open` makes no sense with element being optional, so
    I skipped it.
  * `bst workspace close` will probably need to be revisited when multiple
    projects can own one workspace.
  * `bst workspace reset` will happily delete the directory you're
    currently in, requiring you to `cd $PWD` to see the contents of your
    directory.
    I could exclude the top-level directory of the workspace being
    deleted, but it is entirely valid to run workspace commands from deeper
    in the workspace.
  * I'm not sure if `bst source-bundle` is working correctly, but I don't
    think my changes are responsible.
  

• 989af048
  by Jonathan Maw at 2018-11-27T17:09:06Z

  tests: Add tests for guessing element names
  
  Note: This omits tests of `bst shell` because a command-line can't be
  passed without also specifying an element.
  

10 changed files:

• NEWS
• buildstream/_context.py
• buildstream/_frontend/app.py
• buildstream/_frontend/cli.py
• buildstream/_project.py
• buildstream/_stream.py
• buildstream/_workspaces.py
• buildstream/data/userconfig.yaml
• tests/frontend/workspace.py
• tests/integration/shell.py

Changes:

NEWS

---

...	...	@@ -70,6 +70,9 @@ buildstream 1.3.1
70	70	o Add sandbox API for command batching and use it for build, script, and
71	71	compose elements.
72	72
	73	+ o Opening a workspace now creates a .bstproject.yaml file that allows buildstream
	74	+ commands to be run from a workspace that is not inside a project.
	75	+
73	76
74	77	=================
75	78	buildstream 1.1.5

buildstream/_context.py

---

...	...	@@ -31,7 +31,7 @@ from ._exceptions import LoadError, LoadErrorReason, BstError
31	31	from ._message import Message, MessageType
32	32	from ._profile import Topics, profile_start, profile_end
33	33	from ._artifactcache import ArtifactCache
34		-from ._workspaces import Workspaces
	34	+from ._workspaces import Workspaces, WorkspaceProjectCache
35	35	from .plugin import _plugin_lookup
36	36
37	37
...	...	@@ -46,9 +46,13 @@ from .plugin import _plugin_lookup
46	46	# verbosity levels and basically anything pertaining to the context
47	47	# in which BuildStream was invoked.
48	48	#
	49	+# Args:
	50	+# workspace_project_cache (WorkspaceProjectCache): A WorkspaceProjectCache
	51	+# for this invocation
	52	+#
49	53	class Context():
50	54
51		- def __init__(self):
	55	+ def __init__(self, workspace_project_cache=None):
52	56
53	57	# Filename indicating which configuration file was used, or None for the defaults
54	58	self.config_origin = None
...	...	@@ -121,6 +125,10 @@ class Context():
121	125	# remove a workspace directory.
122	126	self.prompt_workspace_close_remove_dir = None
123	127
	128	+ # Boolean, whether we double-check with the user that they meant to
	129	+ # close the workspace when they're using it to access the project.
	130	+ self.prompt_workspace_close_project_inaccessible = None
	131	+
124	132	# Boolean, whether we double-check with the user that they meant to do
125	133	# a hard reset of a workspace, potentially losing changes.
126	134	self.prompt_workspace_reset_hard = None
...	...	@@ -139,6 +147,7 @@ class Context():
139	147	self._projects = []
140	148	self._project_overrides = {}
141	149	self._workspaces = None
	150	+ self._workspace_project_cache = workspace_project_cache or WorkspaceProjectCache()
142	151	self._log_handle = None
143	152	self._log_filename = None
144	153
...	...	@@ -248,12 +257,15 @@ class Context():
248	257	defaults, Mapping, 'prompt')
249	258	_yaml.node_validate(prompt, [
250	259	'auto-init', 'really-workspace-close-remove-dir',
	260	+ 'really-workspace-close-project-inaccessible',
251	261	'really-workspace-reset-hard',
252	262	])
253	263	self.prompt_auto_init = _node_get_option_str(
254	264	prompt, 'auto-init', ['ask', 'no']) == 'ask'
255	265	self.prompt_workspace_close_remove_dir = _node_get_option_str(
256	266	prompt, 'really-workspace-close-remove-dir', ['ask', 'yes']) == 'ask'
	267	+ self.prompt_workspace_close_project_inaccessible = _node_get_option_str(
	268	+ prompt, 'really-workspace-close-project-inaccessible', ['ask', 'yes']) == 'ask'
257	269	self.prompt_workspace_reset_hard = _node_get_option_str(
258	270	prompt, 'really-workspace-reset-hard', ['ask', 'yes']) == 'ask'
259	271
...	...	@@ -310,6 +322,16 @@ class Context():
310	322	def get_workspaces(self):
311	323	return self._workspaces
312	324
	325	+ # get_workspace_project_cache():
	326	+ #
	327	+ # Return the WorkspaceProjectCache object used for this BuildStream invocation
	328	+ #
	329	+ # Returns:
	330	+ # (WorkspaceProjectCache): The WorkspaceProjectCache object
	331	+ #
	332	+ def get_workspace_project_cache(self):
	333	+ return self._workspace_project_cache
	334	+
313	335	# get_overrides():
314	336	#
315	337	# Fetch the override dictionary for the active project. This returns

buildstream/_frontend/app.py

---

...	...	@@ -39,6 +39,7 @@ from .._stream import Stream
39	39	from .._versions import BST_FORMAT_VERSION
40	40	from .. import _yaml
41	41	from .._scheduler import ElementJob
	42	+from .._workspaces import WorkspaceProjectCache
42	43
43	44	# Import frontend assets
44	45	from . import Profile, LogLine, Status
...	...	@@ -79,6 +80,7 @@ class App():
79	80	self._fail_messages = {} # Failure messages by unique plugin id
80	81	self._interactive_failures = None # Whether to handle failures interactively
81	82	self._started = False # Whether a session has started
	83	+ self._workspace_project_cache = WorkspaceProjectCache() # A collection of workspace local data
82	84
83	85	# UI Colors Profiles
84	86	self._content_profile = Profile(fg='yellow')
...	...	@@ -164,7 +166,7 @@ class App():
164	166	# Load the Context
165	167	#
166	168	try:
167		- self.context = Context()
	169	+ self.context = Context(self._workspace_project_cache)
168	170	self.context.load(config)
169	171	except BstError as e:
170	172	self._error_exit(e, "Error loading user configuration")
...	...	@@ -402,6 +404,20 @@ class App():
402	404	if self.stream:
403	405	self.stream.cleanup()
404	406
	407	+ # guess_element()
	408	+ #
	409	+ # Attempts to interpret which element the user intended to run commands on
	410	+ #
	411	+ # Returns:
	412	+ # (str) The name of the element, or an empty string
	413	+ def guess_element(self):
	414	+ directory = self._main_options['directory']
	415	+ workspace_project = self._workspace_project_cache.get(directory)
	416	+ if workspace_project:
	417	+ return workspace_project.get_default_element()
	418	+ else:
	419	+ return ""
	420	+
405	421	############################################################
406	422	# Abstract Class Methods #
407	423	############################################################

buildstream/_frontend/cli.py

---

...	...	@@ -312,6 +312,12 @@ def build(app, elements, all_, track_, track_save, track_all, track_except, trac
312	312	if track_save:
313	313	click.echo("WARNING: --track-save is deprecated, saving is now unconditional", err=True)
314	314
	315	+ if not all_ and not elements:
	316	+ # Attempt to divine the element from the workspace you're in
	317	+ guessed_target = app.guess_element()
	318	+ if guessed_target:
	319	+ elements = (guessed_target,)
	320	+
315	321	if track_all:
316	322	track_ = elements
317	323
...	...	@@ -366,6 +372,12 @@ def fetch(app, elements, deps, track_, except_, track_cross_junctions):
366	372	"Since tracking modifies the build plan, all elements will be tracked.", err=True)
367	373	deps = PipelineSelection.ALL
368	374
	375	+ if not elements:
	376	+ # Attempt to divine the element from the workspace you're in
	377	+ guessed_target = app.guess_element()
	378	+ if guessed_target:
	379	+ elements = (guessed_target,)
	380	+
369	381	with app.initialized(session_name="Fetch"):
370	382	app.stream.fetch(elements,
371	383	selection=deps,
...	...	@@ -402,6 +414,12 @@ def track(app, elements, deps, except_, cross_junctions):
402	414	none: No dependencies, just the specified elements
403	415	all: All dependencies of all specified elements
404	416	"""
	417	+ if not elements:
	418	+ # Attempt to divine the element from the workspace you're in
	419	+ guessed_target = app.guess_element()
	420	+ if guessed_target:
	421	+ elements = (guessed_target,)
	422	+
405	423	with app.initialized(session_name="Track"):
406	424	# Substitute 'none' for 'redirect' so that element redirections
407	425	# will be done
...	...	@@ -438,6 +456,12 @@ def pull(app, elements, deps, remote):
438	456	none: No dependencies, just the element itself
439	457	all: All dependencies
440	458	"""
	459	+ if not elements:
	460	+ # Attempt to divine the element from the workspace you're in
	461	+ guessed_target = app.guess_element()
	462	+ if guessed_target:
	463	+ elements = (guessed_target,)
	464	+
441	465	with app.initialized(session_name="Pull"):
442	466	app.stream.pull(elements, selection=deps, remote=remote)
443	467
...	...	@@ -466,6 +490,11 @@ def push(app, elements, deps, remote):
466	490	none: No dependencies, just the element itself
467	491	all: All dependencies
468	492	"""
	493	+ if not elements:
	494	+ # Attempt to divine the element from the workspace you're in
	495	+ guessed_target = app.guess_element()
	496	+ if guessed_target:
	497	+ elements = (guessed_target,)
469	498	with app.initialized(session_name="Push"):
470	499	app.stream.push(elements, selection=deps, remote=remote)
471	500
...	...	@@ -536,6 +565,12 @@ def show(app, elements, deps, except_, order, format_):
536	565	bst show target.bst --format \\
537	566	$'---------- %{name} ----------\\n%{vars}'
538	567	"""
	568	+ if not elements:
	569	+ # Attempt to divine the element from the workspace you're in
	570	+ guessed_target = app.guess_element()
	571	+ if guessed_target:
	572	+ elements = (guessed_target,)
	573	+
539	574	with app.initialized():
540	575	dependencies = app.stream.load_selection(elements,
541	576	selection=deps,
...	...	@@ -565,7 +600,7 @@ def show(app, elements, deps, except_, order, format_):
565	600	help="Mount a file or directory into the sandbox")
566	601	@click.option('--isolate', is_flag=True, default=False,
567	602	help='Create an isolated build sandbox')
568		-@click.argument('element',
	603	+@click.argument('element', required=False,
569	604	type=click.Path(readable=False))
570	605	@click.argument('command', type=click.STRING, nargs=-1)
571	606	@click.pass_obj
...	...	@@ -596,6 +631,14 @@ def shell(app, element, sysroot, mount, isolate, build_, command):
596	631	scope = Scope.RUN
597	632
598	633	with app.initialized():
	634	+ if not element:
	635	+ # Attempt to divine the element from the workspace you're in
	636	+ guessed_target = app.guess_element()
	637	+ if guessed_target:
	638	+ element = guessed_target
	639	+ else:
	640	+ raise AppError('Error: Missing argument "ELEMENT".')
	641	+
599	642	dependencies = app.stream.load_selection((element,), selection=PipelineSelection.NONE)
600	643	element = dependencies[0]
601	644	prompt = app.shell_prompt(element)
...	...	@@ -633,14 +676,27 @@ def shell(app, element, sysroot, mount, isolate, build_, command):
633	676	help="Create a tarball from the artifact contents instead "
634	677	"of a file tree. If LOCATION is '-', the tarball "
635	678	"will be dumped to the standard output.")
636		-@click.argument('element',
	679	+@click.argument('element', required=False,
637	680	type=click.Path(readable=False))
638		-@click.argument('location', type=click.Path())
	681	+@click.argument('location', type=click.Path(), required=False)
639	682	@click.pass_obj
640	683	def checkout(app, element, location, force, deps, integrate, hardlinks, tar):
641	684	"""Checkout a built artifact to the specified location
642	685	"""
643	686
	687	+ if not element and not location:
	688	+ click.echo("ERROR: LOCATION is not specified", err=True)
	689	+ sys.exit(-1)
	690	+
	691	+ if element and not location:
	692	+ # Nasty hack to get around click's optional args
	693	+ location = element
	694	+ element = app.guess_element()
	695	+
	696	+ if not element:
	697	+ click.echo("ERROR: ELEMENT is not specified", err=True)
	698	+ sys.exit(-1)
	699	+
644	700	if hardlinks and tar:
645	701	click.echo("ERROR: options --hardlinks and --tar conflict", err=True)
646	702	sys.exit(-1)
...	...	@@ -732,8 +788,14 @@ def workspace_close(app, remove_dir, all_, elements):
732	788	"""Close a workspace"""
733	789
734	790	if not (all_ or elements):
735		- click.echo('ERROR: no elements specified', err=True)
736		- sys.exit(-1)
	791	+ # NOTE: I may need to revisit this when implementing multiple projects
	792	+ # opening one workspace.
	793	+ element = app.guess_element()
	794	+ if element:
	795	+ elements = (element,)
	796	+ else:
	797	+ click.echo('ERROR: no elements specified', err=True)
	798	+ sys.exit(-1)
737	799
738	800	with app.initialized():
739	801
...	...	@@ -747,11 +809,18 @@ def workspace_close(app, remove_dir, all_, elements):
747	809
748	810	elements = app.stream.redirect_element_names(elements)
749	811
750		- # Check that the workspaces in question exist
	812	+ # Check that the workspaces in question exist, and that it's safe to
	813	+ # remove them.
751	814	nonexisting = []
752	815	for element_name in elements:
753	816	if not app.stream.workspace_exists(element_name):
754	817	nonexisting.append(element_name)
	818	+ if (app.stream.workspace_is_required(element_name) and app.interactive and
	819	+ app.context.prompt_workspace_close_project_inaccessible):
	820	+ click.echo("Removing '{}' will prevent you from running buildstream commands".format(element_name))
	821	+ if not click.confirm('Are you sure you want to close this workspace?'):
	822	+ click.echo('Aborting', err=True)
	823	+ sys.exit(-1)
755	824	if nonexisting:
756	825	raise AppError("Workspace does not exist", detail="\n".join(nonexisting))
757	826
...	...	@@ -784,7 +853,11 @@ def workspace_reset(app, soft, track_, all_, elements):
784	853	with app.initialized():
785	854
786	855	if not (all_ or elements):
787		- raise AppError('No elements specified to reset')
	856	+ element = app.guess_element()
	857	+ if element:
	858	+ elements = (element,)
	859	+ else:
	860	+ raise AppError('No elements specified to reset')
788	861
789	862	if all_ and not app.stream.workspace_exists():
790	863	raise AppError("No open workspaces to reset")

buildstream/_project.py

---

...	...	@@ -94,8 +94,10 @@ class Project():
94	94	# The project name
95	95	self.name = None
96	96
97		- # The project directory
98		- self.directory = self._find_project_dir(directory)
	97	+ self._context = context # The invocation Context, a private member
	98	+
	99	+ # The project directory, and whether the project was found from an external workspace
	100	+ self.directory, self._required_workspace_element = self._find_project_dir(directory)
99	101
100	102	# Absolute path to where elements are loaded from within the project
101	103	self.element_path = None
...	...	@@ -116,7 +118,6 @@ class Project():
116	118	#
117	119	# Private Members
118	120	#
119		- self._context = context # The invocation Context
120	121
121	122	self._default_mirror = default_mirror # The name of the preferred mirror.
122	123
...	...	@@ -370,6 +371,14 @@ class Project():
370	371
371	372	self._load_second_pass()
372	373
	374	+ # required_workspace_element()
	375	+ #
	376	+ # Returns the element whose workspace is required to load this project,
	377	+ # if any.
	378	+ #
	379	+ def required_workspace_element(self):
	380	+ return self._required_workspace_element
	381	+
373	382	# cleanup()
374	383	#
375	384	# Cleans up resources used loading elements
...	...	@@ -662,18 +671,26 @@ class Project():
662	671	# Raises:
663	672	# LoadError if project.conf is not found
664	673	#
	674	+ # Returns:
	675	+ # (str) - the directory that contains the project, and
	676	+ # (str) - the name of the element required to find the project, or an empty string
	677	+ #
665	678	def _find_project_dir(self, directory):
666		- directory = os.path.abspath(directory)
667		- while not os.path.isfile(os.path.join(directory, _PROJECT_CONF_FILE)):
668		- parent_dir = os.path.dirname(directory)
669		- if directory == parent_dir:
	679	+ workspace_element = ""
	680	+ project_directory = utils._search_upward_for_file(directory, _PROJECT_CONF_FILE)
	681	+ if not project_directory:
	682	+ workspace_project_cache = self._context.get_workspace_project_cache()
	683	+ workspace_project = workspace_project_cache.get(directory)
	684	+ if workspace_project:
	685	+ project_directory = workspace_project.get_default_path()
	686	+ workspace_element = workspace_project.get_default_element()
	687	+ else:
670	688	raise LoadError(
671	689	LoadErrorReason.MISSING_PROJECT_CONF,
672	690	'{} not found in current directory or any of its parent directories'
673	691	.format(_PROJECT_CONF_FILE))
674		- directory = parent_dir
675	692
676		- return directory
	693	+ return project_directory, workspace_element
677	694
678	695	def _load_plugin_factories(self, config, output):
679	696	plugin_source_origins = [] # Origins of custom sources

buildstream/_stream.py

---

...	...	@@ -28,7 +28,7 @@ import tarfile
28	28	from contextlib import contextmanager
29	29	from tempfile import TemporaryDirectory
30	30
31		-from ._exceptions import StreamError, ImplError, BstError, set_last_task_error
	31	+from ._exceptions import StreamError, ImplError, BstError, set_last_task_error, LoadError, LoadErrorReason
32	32	from ._message import Message, MessageType
33	33	from ._scheduler import Scheduler, SchedStatus, TrackQueue, FetchQueue, BuildQueue, PullQueue, PushQueue
34	34	from ._pipeline import Pipeline, PipelineSelection
...	...	@@ -550,6 +550,8 @@ class Stream():
550	550	# So far this function has tried to catch as many issues as possible with out making any changes
551	551	# Now it dose the bits that can not be made atomic.
552	552	targetGenerator = zip(elements, expanded_directories)
	553	+ workspace_project_cache = self._context.get_workspace_project_cache()
	554	+ project = self._context.get_toplevel_project()
553	555	for target, directory in targetGenerator:
554	556	self._message(MessageType.INFO, "Creating workspace for element {}"
555	557	.format(target.name))
...	...	@@ -574,6 +576,10 @@ class Stream():
574	576	with target.timed_activity("Staging sources to {}".format(directory)):
575	577	target._open_workspace()
576	578
	579	+ workspace_project = workspace_project_cache.add(directory, project.directory,
	580	+ target._get_full_name())
	581	+ workspace_project.write()
	582	+
577	583	# Saving the workspace once it is set up means that if the next workspace fails to be created before
578	584	# the configuration gets saved. The successfully created workspace still gets saved.
579	585	workspaces.save_config()
...	...	@@ -601,6 +607,16 @@ class Stream():
601	607	except OSError as e:
602	608	raise StreamError("Could not remove '{}': {}"
603	609	.format(workspace.get_absolute_path(), e)) from e
	610	+ else:
	611	+ workspace_project_cache = self._context.get_workspace_project_cache()
	612	+ try:
	613	+ workspace_project_cache.remove(workspace.get_absolute_path())
	614	+ except LoadError as e:
	615	+ # We might be closing a workspace with a deleted directory
	616	+ if e.reason == LoadErrorReason.MISSING_FILE:
	617	+ pass
	618	+ else:
	619	+ raise
604	620
605	621	# Delete the workspace and save the configuration
606	622	workspaces.delete_workspace(element_name)
...	...	@@ -644,6 +660,8 @@ class Stream():
644	660	for element in elements:
645	661	workspace = workspaces.get_workspace(element._get_full_name())
646	662	workspace_path = workspace.get_absolute_path()
	663	+ workspace_project_cache = self._context.get_workspace_project_cache()
	664	+ workspace_project = workspace_project_cache.get(workspace_path)
647	665	if soft:
648	666	workspace.prepared = False
649	667	self._message(MessageType.INFO, "Reset workspace state for {} at: {}"
...	...	@@ -664,6 +682,8 @@ class Stream():
664	682	with element.timed_activity("Staging sources to {}".format(workspace_path)):
665	683	element._open_workspace()
666	684
	685	+ workspace_project.write()
	686	+
667	687	self._message(MessageType.INFO,
668	688	"Reset workspace for {} at: {}".format(element.name,
669	689	workspace_path))
...	...	@@ -694,6 +714,20 @@ class Stream():
694	714
695	715	return False
696	716
	717	+ # workspace_is_required()
	718	+ #
	719	+ # Checks whether the workspace belonging to element_name is required to
	720	+ # load the project
	721	+ #
	722	+ # Args:
	723	+ # element_name (str): The element whose workspace may be required
	724	+ #
	725	+ # Returns:
	726	+ # (bool): True if the workspace is required
	727	+ def workspace_is_required(self, element_name):
	728	+ required_elm = self._project.required_workspace_element()
	729	+ return required_elm == element_name
	730	+
697	731	# workspace_list
698	732	#
699	733	# Serializes the workspaces and dumps them in YAML to stdout.

buildstream/_workspaces.py

---

...	...	@@ -379,10 +379,15 @@ class Workspace():
379	379	if recalculate or self._key is None:
380	380	fullpath = self.get_absolute_path()
381	381
	382	+ excluded_files = (WORKSPACE_PROJECT_FILE,)
	383	+
382	384	# Get a list of tuples of the the project relative paths and fullpaths
383	385	if os.path.isdir(fullpath):
384	386	filelist = utils.list_relative_paths(fullpath)
385		- filelist = [(relpath, os.path.join(fullpath, relpath)) for relpath in filelist]
	387	+ filelist = [
	388	+ (relpath, os.path.join(fullpath, relpath)) for relpath in filelist
	389	+ if relpath not in excluded_files
	390	+ ]
386	391	else:
387	392	filelist = [(self.get_absolute_path(), fullpath)]
388	393

buildstream/data/userconfig.yaml

---

...	...	@@ -128,6 +128,14 @@ prompt:
128	128	#
129	129	really-workspace-close-remove-dir: ask
130	130
	131	+ # Whether to really proceed with 'bst workspace close' when doing so would
	132	+ # stop them from running bst commands in this workspace.
	133	+ #
	134	+ # ask - Ask the user if they are sure.
	135	+ # yes - Always close, without asking.
	136	+ #
	137	+ really-workspace-close-project-inaccessible: ask
	138	+
131	139	# Whether to really proceed with 'bst workspace reset' doing a hard reset of
132	140	# a workspace, potentially losing changes.
133	141	#

tests/frontend/workspace.py

---

...	...	@@ -31,6 +31,7 @@ import shutil
31	31	import subprocess
32	32	from ruamel.yaml.comments import CommentedSet
33	33	from tests.testutils import cli, create_repo, ALL_REPO_KINDS, wait_for_cache_granularity
	34	+from tests.testutils import create_artifact_share
34	35
35	36	from buildstream import _yaml
36	37	from buildstream._exceptions import ErrorDomain, LoadError, LoadErrorReason
...	...	@@ -615,9 +616,15 @@ def test_list(cli, tmpdir, datafiles):
615	616	@pytest.mark.datafiles(DATA_DIR)
616	617	@pytest.mark.parametrize("kind", repo_kinds)
617	618	@pytest.mark.parametrize("strict", [("strict"), ("non-strict")])
618		-def test_build(cli, tmpdir, datafiles, kind, strict):
	619	+@pytest.mark.parametrize("call_from,guess_element", [
	620	+ ("project", "no-guess"), ("workspace", "guess"), ("workspace", "no-guess")
	621	+])
	622	+def test_build(cli, tmpdir_factory, datafiles, kind, strict, call_from, guess_element):
	623	+ tmpdir = tmpdir_factory.mktemp('')
619	624	element_name, project, workspace = open_workspace(cli, tmpdir, datafiles, kind, False)
620	625	checkout = os.path.join(str(tmpdir), 'checkout')
	626	+ args_dir = ['-C', workspace] if call_from == "workspace" else []
	627	+ args_elm = [element_name] if guess_element == "no-guess" else []
621	628
622	629	# Modify workspace
623	630	shutil.rmtree(os.path.join(workspace, 'usr', 'bin'))
...	...	@@ -640,15 +647,14 @@ def test_build(cli, tmpdir, datafiles, kind, strict):
640	647	# Build modified workspace
641	648	assert cli.get_element_state(project, element_name) == 'buildable'
642	649	assert cli.get_element_key(project, element_name) == "{:?<64}".format('')
643		- result = cli.run(project=project, args=['build', element_name])
	650	+ result = cli.run(project=project, args=args_dir + ['build'] + args_elm)
644	651	result.assert_success()
645	652	assert cli.get_element_state(project, element_name) == 'cached'
646	653	assert cli.get_element_key(project, element_name) != "{:?<64}".format('')
647	654
648	655	# Checkout the result
649		- result = cli.run(project=project, args=[
650		- 'checkout', element_name, checkout
651		- ])
	656	+ result = cli.run(project=project,
	657	+ args=args_dir + ['checkout'] + args_elm + [checkout])
652	658	result.assert_success()
653	659
654	660	# Check that the pony.conf from the modified workspace exists
...	...	@@ -1055,3 +1061,144 @@ def test_multiple_failed_builds(cli, tmpdir, datafiles):
1055	1061	result = cli.run(project=project, args=["build", element_name])
1056	1062	assert "BUG" not in result.stderr
1057	1063	assert cli.get_element_state(project, element_name) != "cached"
	1064	+
	1065	+
	1066	+@pytest.mark.datafiles(DATA_DIR)
	1067	+@pytest.mark.parametrize("guess_element", [("guess"), ("no-guess")])
	1068	+def test_external_fetch(cli, datafiles, tmpdir_factory, guess_element):
	1069	+ # Fetching from a workspace outside a project doesn't fail horribly
	1070	+ tmpdir = tmpdir_factory.mktemp('')
	1071	+ element_name, project, workspace = open_workspace(cli, tmpdir, datafiles, "git", False)
	1072	+ arg_elm = [element_name] if guess_element == "no-guess" else []
	1073	+
	1074	+ result = cli.run(project=project, args=['-C', workspace, 'fetch'] + arg_elm)
	1075	+ result.assert_success()
	1076	+
	1077	+ # We already fetched it by opening the workspace, but we're also checking
	1078	+ # `bst show` works here
	1079	+ result = cli.run(project=project,
	1080	+ args=['-C', workspace, 'show', '--deps', 'none', '--format', '%{state}'] + arg_elm)
	1081	+ result.assert_success()
	1082	+ assert result.output.strip() == 'buildable'
	1083	+
	1084	+
	1085	+@pytest.mark.datafiles(DATA_DIR)
	1086	+@pytest.mark.parametrize("guess_element", [("guess"), ("no-guess")])
	1087	+def test_external_push_pull(cli, datafiles, tmpdir_factory, guess_element):
	1088	+ # Pushing and pulling to/from an artifact cache works from an external workspace
	1089	+ tmpdir = tmpdir_factory.mktemp('')
	1090	+ element_name, project, workspace = open_workspace(cli, tmpdir, datafiles, "git", False)
	1091	+ args_elm = [element_name] if guess_element == "no-guess" else []
	1092	+
	1093	+ with create_artifact_share(os.path.join(str(tmpdir), 'artifactshare')) as share:
	1094	+ result = cli.run(project=project, args=['-C', workspace, 'build', element_name])
	1095	+ result.assert_success()
	1096	+
	1097	+ cli.configure({
	1098	+ 'artifacts': {'url': share.repo, 'push': True}
	1099	+ })
	1100	+
	1101	+ result = cli.run(project=project, args=['-C', workspace, 'push'] + args_elm)
	1102	+ result.assert_success()
	1103	+
	1104	+ result = cli.run(project=project, args=['-C', workspace, 'pull', '--deps', 'all'] + args_elm)
	1105	+ result.assert_success()
	1106	+
	1107	+
	1108	+@pytest.mark.datafiles(DATA_DIR)
	1109	+@pytest.mark.parametrize("guess_element", [("guess"), ("no-guess")])
	1110	+def test_external_track(cli, datafiles, tmpdir_factory, guess_element):
	1111	+ # Tracking does not get horribly confused
	1112	+ tmpdir = tmpdir_factory.mktemp('')
	1113	+ element_name, project, workspace = open_workspace(cli, tmpdir, datafiles, "git", True)
	1114	+ args_elm = [element_name] if guess_element == "no-guess" else []
	1115	+
	1116	+ # The workspace is necessarily already tracked, so we only care that
	1117	+ # there's no weird errors.
	1118	+ result = cli.run(project=project, args=['-C', workspace, 'track'] + args_elm)
	1119	+ result.assert_success()
	1120	+
	1121	+
	1122	+@pytest.mark.datafiles(DATA_DIR)
	1123	+def test_external_open_other(cli, datafiles, tmpdir_factory):
	1124	+ # >From inside an external workspace, open another workspace
	1125	+ tmpdir1 = tmpdir_factory.mktemp('')
	1126	+ tmpdir2 = tmpdir_factory.mktemp('')
	1127	+ # Making use of the assumption that it's the same project in both invocations of open_workspace
	1128	+ alpha_element, project, alpha_workspace = open_workspace(cli, tmpdir1, datafiles, "git", False, suffix="-alpha")
	1129	+ beta_element, _, beta_workspace = open_workspace(cli, tmpdir2, datafiles, "git", False, suffix="-beta")
	1130	+
	1131	+ # Closing the other element first, because I'm too lazy to create an
	1132	+ # element without opening it
	1133	+ result = cli.run(project=project, args=['workspace', 'close', beta_element])
	1134	+ result.assert_success()
	1135	+
	1136	+ result = cli.run(project=project, args=[
	1137	+ '-C', alpha_workspace, 'workspace', 'open', '--force', '--directory', beta_workspace, beta_element
	1138	+ ])
	1139	+ result.assert_success()
	1140	+
	1141	+
	1142	+@pytest.mark.datafiles(DATA_DIR)
	1143	+def test_external_close_other(cli, datafiles, tmpdir_factory):
	1144	+ # >From inside an external workspace, close the other workspace
	1145	+ tmpdir1 = tmpdir_factory.mktemp('')
	1146	+ tmpdir2 = tmpdir_factory.mktemp('')
	1147	+ # Making use of the assumption that it's the same project in both invocations of open_workspace
	1148	+ alpha_element, project, alpha_workspace = open_workspace(cli, tmpdir1, datafiles, "git", False, suffix="-alpha")
	1149	+ beta_element, _, beta_workspace = open_workspace(cli, tmpdir2, datafiles, "git", False, suffix="-beta")
	1150	+
	1151	+ result = cli.run(project=project, args=['-C', alpha_workspace, 'workspace', 'close', beta_element])
	1152	+ result.assert_success()
	1153	+
	1154	+
	1155	+@pytest.mark.datafiles(DATA_DIR)
	1156	+@pytest.mark.parametrize("guess_element", [("guess"), ("no-guess")])
	1157	+def test_external_close_self(cli, datafiles, tmpdir_factory, guess_element):
	1158	+ # >From inside an external workspace, close it
	1159	+ tmpdir1 = tmpdir_factory.mktemp('')
	1160	+ tmpdir2 = tmpdir_factory.mktemp('')
	1161	+ # Making use of the assumption that it's the same project in both invocations of open_workspace
	1162	+ alpha_element, project, alpha_workspace = open_workspace(cli, tmpdir1, datafiles, "git", False, suffix="-alpha")
	1163	+ beta_element, _, beta_workspace = open_workspace(cli, tmpdir2, datafiles, "git", False, suffix="-beta")
	1164	+ arg_elm = [alpha_element] if guess_element == "no-guess" else []
	1165	+
	1166	+ result = cli.run(project=project, args=['-C', alpha_workspace, 'workspace', 'close'] + arg_elm)
	1167	+ result.assert_success()
	1168	+
	1169	+
	1170	+@pytest.mark.datafiles(DATA_DIR)
	1171	+def test_external_reset_other(cli, datafiles, tmpdir_factory):
	1172	+ tmpdir1 = tmpdir_factory.mktemp('')
	1173	+ tmpdir2 = tmpdir_factory.mktemp('')
	1174	+ # Making use of the assumption that it's the same project in both invocations of open_workspace
	1175	+ alpha_element, project, alpha_workspace = open_workspace(cli, tmpdir1, datafiles, "git", False, suffix="-alpha")
	1176	+ beta_element, _, beta_workspace = open_workspace(cli, tmpdir2, datafiles, "git", False, suffix="-beta")
	1177	+
	1178	+ result = cli.run(project=project, args=['-C', alpha_workspace, 'workspace', 'reset', beta_element])
	1179	+ result.assert_success()
	1180	+
	1181	+
	1182	+@pytest.mark.datafiles(DATA_DIR)
	1183	+@pytest.mark.parametrize("guess_element", [("guess"), ("no-guess")])
	1184	+def test_external_reset_self(cli, datafiles, tmpdir, guess_element):
	1185	+ element, project, workspace = open_workspace(cli, tmpdir, datafiles, "git", False)
	1186	+ arg_elm = [element] if guess_element == "no-guess" else []
	1187	+
	1188	+ # Command succeeds
	1189	+ result = cli.run(project=project, args=['-C', workspace, 'workspace', 'reset'] + arg_elm)
	1190	+ result.assert_success()
	1191	+
	1192	+ # Successive commands still work (i.e. .bstproject.yaml hasn't been deleted)
	1193	+ result = cli.run(project=project, args=['-C', workspace, 'workspace', 'list'])
	1194	+ result.assert_success()
	1195	+
	1196	+
	1197	+@pytest.mark.datafiles(DATA_DIR)
	1198	+def test_external_list(cli, datafiles, tmpdir_factory):
	1199	+ tmpdir = tmpdir_factory.mktemp('')
	1200	+ # Making use of the assumption that it's the same project in both invocations of open_workspace
	1201	+ element, project, workspace = open_workspace(cli, tmpdir, datafiles, "git", False)
	1202	+
	1203	+ result = cli.run(project=project, args=['-C', workspace, 'workspace', 'list'])
	1204	+ result.assert_success()

tests/integration/shell.py

---

...	...	@@ -353,3 +353,39 @@ def test_integration_devices(cli, tmpdir, datafiles):
353	353
354	354	result = execute_shell(cli, project, ["true"], element=element_name)
355	355	assert result.exit_code == 0
	356	+
	357	+
	358	+# Test that a shell can be opened from an external workspace
	359	+@pytest.mark.datafiles(DATA_DIR)
	360	+@pytest.mark.parametrize("build_shell", [("build"), ("nobuild")])
	361	+@pytest.mark.parametrize("guess_element", [("guess"), ("no-guess")])
	362	+@pytest.mark.skipif(IS_LINUX and not HAVE_BWRAP, reason='Only available with bubblewrap on Linux')
	363	+def test_integration_external_workspace(cli, tmpdir_factory, datafiles, build_shell, guess_element):
	364	+ tmpdir = tmpdir_factory.mktemp("")
	365	+ project = os.path.join(datafiles.dirname, datafiles.basename)
	366	+ element_name = 'autotools/amhello.bst'
	367	+ workspace_dir = os.path.join(str(tmpdir), 'workspace')
	368	+
	369	+ if guess_element == "guess":
	370	+ # Mutate the project.conf to use a default shell command
	371	+ project_file = os.path.join(project, 'project.conf')
	372	+ config_text = "shell:\n"\
	373	+ " command: ['true']\n"
	374	+ with open(project_file, 'a') as f:
	375	+ f.write(config_text)
	376	+
	377	+ result = cli.run(project=project, args=[
	378	+ 'workspace', 'open', '--directory', workspace_dir, element_name
	379	+ ])
	380	+ result.assert_success()
	381	+
	382	+ result = cli.run(project=project, args=['-C', workspace_dir, 'build', element_name])
	383	+ result.assert_success()
	384	+
	385	+ command = ['-C', workspace_dir, 'shell']
	386	+ if build_shell == 'build':
	387	+ command.append('--build')
	388	+ if guess_element == "no-guess":
	389	+ command.extend([element_name, '--', 'true'])
	390	+ result = cli.run(project=project, cwd=workspace_dir, args=command)
	391	+ result.assert_success()

—
View it on GitLab.
You're receiving this email because of your account on gitlab.com. If you'd like to receive fewer emails, you can adjust your notification settings.