[Notes] [Git][BuildStream/buildstream][tristan/build-docs] 5 commits: __main__.py: Adding private entry point so we can run BuildStream without installing

[Tristan Van Berkom <gitlab mg gitlab com>]

Title: GitLab

Tristan Van Berkom pushed to branch tristan/build-docs at BuildStream / buildstream

Commits:

• bc63f278
  by Tristan Van Berkom at 2018-08-05T10:07:46Z

  __main__.py: Adding private entry point so we can run BuildStream without installing
  
  This is needed to build docs without installing BuildStream.
  

• 1618f405
  by Tristan Van Berkom at 2018-08-05T10:07:46Z

  doc: Fix junctions user guide to refer to doc/sessions, not doc/sessions-stored.
  

• 9e573eeb
  by Tristan Van Berkom at 2018-08-05T10:07:46Z

  doc: Some modifications to the bst2html program
  
    o Added feature to allow running shell commands
  
    o Removed unused codepaths to launch bst commands through the script,
      this program is anyway only used to parse and run the session files.
  
    o Fixed previous patch to run BuildStream uninstalled with
      the subprocess module, as running it in the same interpretor
      causes a bunch of problems.
  

• 19d2a072
  by Tristan Van Berkom at 2018-08-05T10:07:46Z

  doc/sessions/developing.run: Fix the session to actually run `patch` as a shell command
  
  Since we added support for shell commands in bst2html, this can now be fixed.
  
  This is a part of issue #553
  

• 9b937a97
  by Tristan Van Berkom at 2018-08-05T10:07:46Z

  HACKING.rst: It is not longer required to install BuildStream to build the docs.
  

6 changed files:

• HACKING.rst
• + buildstream/__main__.py
• doc/Makefile
• doc/bst2html.py
• doc/sessions/developing.run
• doc/source/advanced-features/junction-elements.rst

Changes:

HACKING.rst

---

...	...	@@ -261,9 +261,6 @@ using pip or some other mechanism::
261	261	# Additional optional dependencies required
262	262	pip3 install --user arpy
263	263
264		-Furthermore, the documentation build requires that BuildStream itself
265		-be installed, as it will be used in the process of generating its docs.
266		-
267	264	To build the documentation, just run the following::
268	265
269	266	make -C doc

buildstream/__main__.py

---

	1	+##################################################################
	2	+# Private Entry Point #
	3	+##################################################################
	4	+#
	5	+# This allows running the cli when BuildStream is uninstalled,
	6	+# as long as BuildStream repo is in PYTHONPATH, one can run it
	7	+# with:
	8	+#
	9	+# python3 -m buildstream [program args]
	10	+#
	11	+# This is used when we need to run BuildStream before installing,
	12	+# like when we build documentation.
	13	+#
	14	+if __name__ == '__main__':
	15	+ # pylint: disable=no-value-for-parameter
	16	+ from ._frontend.cli import cli
	17	+ cli()

doc/Makefile

---

...	...	@@ -105,7 +105,7 @@ sessions-prep:
105	105	#
106	106	sessions: sessions-prep
107	107	for file in $(wildcard sessions/*.run); do \
108		- PYTHONPATH=$(PYTHONPATH) $(BST2HTML) $(BST2HTMLOPTS) --description $$file; \
	108	+ PYTHONPATH=$(PYTHONPATH) $(BST2HTML) $(BST2HTMLOPTS) $$file; \
109	109	done
110	110
111	111	sessions-clean:

doc/bst2html.py

---

...	...	@@ -37,7 +37,6 @@ import click
37	37
38	38	from buildstream import _yaml
39	39	from buildstream import utils
40		-from buildstream._frontend import cli as bst_cli
41	40	from buildstream._exceptions import BstError
42	41
43	42
...	...	@@ -176,20 +175,6 @@ def ansi2html(text, palette='solarized'):
176	175	return sub
177	176
178	177
179		-@contextmanager
180		-def capture_stdout_stderr():
181		- from io import StringIO
182		-
183		- capture = StringIO()
184		- oldstdout, sys.stdout = sys.stdout, capture
185		- oldstderr, sys.stderr = sys.stderr, capture
186		-
187		- yield capture
188		-
189		- sys.stdout = oldstdout
190		- sys.stderr = oldstderr
191		-
192		-
193	178	# workdir()
194	179	#
195	180	# Sets up a new temp directory with a config file
...	...	@@ -219,7 +204,7 @@ def workdir(source_cache=None):
219	204	yield (tempdir, bst_config_file, source_cache)
220	205
221	206
222		-# run_command()
	207	+# run_bst_command()
223	208	#
224	209	# Runs a command
225	210	#
...	...	@@ -231,18 +216,33 @@ def workdir(source_cache=None):
231	216	# Returns:
232	217	# (str): The colorized combined stdout/stderr of BuildStream
233	218	#
234		-def run_command(config_file, directory, command):
235		- click.echo("Running command in directory '{}': bst {}".format(directory, command), err=True)
	219	+def run_bst_command(config_file, directory, command):
	220	+ click.echo("Running bst command in directory '{}': bst {}".format(directory, command), err=True)
236	221
237		- args = ['--colors', '--config', config_file, '--directory', directory] + shlex.split(command)
	222	+ argv = ['python3', '-m', 'buildstream', '--colors', '--config', config_file] + shlex.split(command)
	223	+ p = subprocess.Popen(argv, cwd=directory, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
	224	+ out, _ = p.communicate()
	225	+ return out.decode('utf-8').strip()
238	226
239		- with capture_stdout_stderr() as capture:
240		- bst_cli.main(args=args, prog_name=bst_cli.name)
241	227
242		- capture.flush()
243		- out = capture.getvalue()
	228	+# run_shell_command()
	229	+#
	230	+# Runs a command
	231	+#
	232	+# Args:
	233	+# directory (str): The project directory
	234	+# command (str): A shell command
	235	+#
	236	+# Returns:
	237	+# (str): The combined stdout/stderr of the shell command
	238	+#
	239	+def run_shell_command(directory, command):
	240	+ click.echo("Running shell command in directory '{}': {}".format(directory, command), err=True)
244	241
245		- return out.strip()
	242	+ argv = shlex.split(command)
	243	+ p = subprocess.Popen(argv, cwd=directory, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
	244	+ out, _ = p.communicate()
	245	+ return out.decode('utf-8').strip()
246	246
247	247
248	248	# generate_html
...	...	@@ -393,12 +393,18 @@ def run_session(description, tempdir, source_cache, palette, config_file, force)
393	393	# Get the command string
394	394	command_str = _yaml.node_get(command, str, 'command')
395	395
	396	+ # Check whether this is a shell command and not a bst command
	397	+ is_shell = _yaml.node_get(command, bool, 'shell', default_value=False)
	398	+
396	399	# Check if there is fake output
397	400	command_fake_output = _yaml.node_get(command, str, 'fake-output', default_value=None)
398	401
399	402	# Run the command, or just use the fake output
400	403	if command_fake_output is None:
401		- command_out = run_command(config_file, directory, command_str)
	404	+ if is_shell:
	405	+ command_out = run_shell_command(directory, command_str)
	406	+ else:
	407	+ command_out = run_bst_command(config_file, directory, command_str)
402	408	else:
403	409	command_out = command_fake_output
404	410
...	...	@@ -434,14 +440,8 @@ def run_session(description, tempdir, source_cache, palette, config_file, force)
434	440	@click.option('--palette', '-p', default='tango',
435	441	type=click.Choice(['solarized', 'solarized-xterm', 'tango', 'xterm', 'console']),
436	442	help="Selects a palette for the output style")
437		-@click.option('--output', '-o',
438		- type=click.Path(file_okay=True, dir_okay=False, writable=True),
439		- help="A file to store the output")
440		-@click.option('--description', '-d',
441		- type=click.Path(file_okay=True, dir_okay=False, readable=True),
442		- help="A file describing what to do")
443		-@click.argument('command', type=click.STRING, nargs=-1)
444		-def run_bst(directory, force, source_cache, description, palette, output, command):
	443	+@click.argument('description', click.Path(file_okay=True, dir_okay=False, readable=True))
	444	+def run_bst(directory, force, source_cache, description, palette):
445	445	"""Run a bst command and capture stdout/stderr in html
446	446
447	447	This command normally takes a description yaml file, see the HACKING
...	...	@@ -450,45 +450,8 @@ def run_bst(directory, force, source_cache, description, palette, output, comman
450	450	if not source_cache and os.environ.get('BST_SOURCE_CACHE'):
451	451	source_cache = os.environ['BST_SOURCE_CACHE']
452	452
453		- if output is not None and not check_needs_build(None, output, force=force):
454		- click.echo("No need to rebuild {}".format(output))
455		- return 0
456		-
457	453	with workdir(source_cache=source_cache) as (tempdir, config_file, source_cache):
458		-
459		- if description:
460		- run_session(description, tempdir, source_cache, palette, config_file, force)
461		- return 0
462		-
463		- # Run a command specified on the CLI
464		- #
465		- if not directory:
466		- directory = os.getcwd()
467		- else:
468		- directory = os.path.abspath(directory)
469		- directory = os.path.realpath(directory)
470		-
471		- if not command:
472		- command = []
473		- command_str = " ".join(command)
474		-
475		- # Run the command
476		- #
477		- command_out = run_command(config_file, directory, command_str)
478		-
479		- # Generate a nice html div for this output
480		- #
481		- converted = generate_html(command_out, directory, config_file,
482		- source_cache, tempdir, palette,
483		- command_str)
484		-
485		- if output is None:
486		- click.echo(converted)
487		- else:
488		- outdir = os.path.dirname(output)
489		- os.makedirs(outdir, exist_ok=True)
490		- with open(output, 'wb') as f:
491		- f.write(converted.encode('utf-8'))
	454	+ run_session(description, tempdir, source_cache, palette, config_file, force)
492	455
493	456	return 0
494	457

doc/sessions/developing.run

---

...	...	@@ -16,7 +16,8 @@ commands:
16	16
17	17	# Apply a patch in the workspace
18	18	- directory: ../examples/developing/
19		- command: show hello.bst; patch workspace_hello/files/src/hello.c update.patch;
	19	+ shell: True
	20	+ command: patch workspace_hello/hello.c update.patch
20	21
21	22	# Rebuild
22	23	- directory: ../examples/developing/

doc/source/advanced-features/junction-elements.rst

---

...	...	@@ -41,7 +41,7 @@ This element consists of a script which calls hello.bst's hello command.
41	41	Building callHello.bst,
42	42
43	43	.. raw:: html
44		- :file: ../sessions-stored/junctions-build.html
	44	+ :file: ../sessions/junctions-build.html
45	45
46	46	You can see that the hello.bst element and it's dependencies from the autotools
47	47	project have been build as part of the pipeline for callHello.bst.
...	...	@@ -49,17 +49,18 @@ project have been build as part of the pipeline for callHello.bst.
49	49	We can now invoke `bst shell`
50	50
51	51	.. raw:: html
52		- :file: ../sessions-stored/junctions-shell.html
	52	+ :file: ../sessions/junctions-shell.html
53	53
54	54	This runs the script files/callHello.sh which will makes use of the hello command from the hello.bst element in the autotools project.
55	55
	56	+
56	57	Cross-junction workspaces
57	58	-------------------------
58	59	You can open workspaces for elements in the project refered to by the junction
59	60	using the syntax `bst open ${junction-name}:{element-name}`. In this example,
60	61
61	62	.. raw:: html
62		- :file: ../sessions-stored/junctions-workspace-open.html
	63	+ :file: ../sessions/junctions-workspace-open.html
63	64
64	65	This has opened a workspace for the hello.bst element from the autotools project.
65	66	This workspace can now be used as normal.

—
View it on GitLab.
You're receiving this email because of your account on gitlab.com. If you'd like to receive fewer emails, you can adjust your notification settings.