[Notes] [Git][BuildStream/buildstream][chandan/update-requirements-one-liner] 5 commits: Use collections.abc for Mapping, Iterable

[Chandan Singh <gitlab mg gitlab com>]

Title: GitLab

Chandan Singh pushed to branch chandan/update-requirements-one-liner at BuildStream / buildstream

Commits:

• e15278d7
  by Thomas Coldrick at 2019-01-03T22:14:01Z

  Use collections.abc for Mapping, Iterable
  
  In _yaml.py and _frontend/complete.py we were getting pylint warnings
  for using collections.Mapping and collections.Iterable, which are
  abstract classes now provided from collections.abc. This patch just
  uses the classes from the right place.
  

• a85da591
  by Tristan Van Berkom at 2019-01-03T22:36:16Z

  Merge branch 'coldtom/collections' into 'master'
  
  Use collections.abc for Mapping, Iterable
  
  Closes #831
  
  See merge request BuildStream/buildstream!1020

• 390ecdfd
  by Chandan Singh at 2019-01-03T23:05:50Z

  Rename "tools" directory to "requirements"
  
  This seems like a better name for the directory, as it more closely
  describes the purpose of its contents.
  

• b0ea897b
  by Chandan Singh at 2019-01-03T23:05:50Z

  Add Makefile to update requirements files
  

• 95efd278
  by Chandan Singh at 2019-01-03T23:05:50Z

  CONTRIBUTING.rst: Add instructions to update requirements files
  
  Split the "The MANIFEST.in and setup.py" section in two:
  "Managing data files" and "Updating BuildStream's Python dependencies".
  
  Briefly explain the layout of `requirements` directory and add
  instructions to use the Makefile added in the last commit.
  

13 changed files:

• CONTRIBUTING.rst
• MANIFEST.in
• buildstream/_frontend/complete.py
• buildstream/_yaml.py
• + requirements/Makefile
• tools/dev-requirements.in → requirements/dev-requirements.in
• tools/dev-requirements.txt → requirements/dev-requirements.txt
• tools/plugin-requirements.in → requirements/plugin-requirements.in
• tools/plugin-requirements.txt → requirements/plugin-requirements.txt
• tools/requirements.in → requirements/requirements.in
• tools/requirements.txt → requirements/requirements.txt
• setup.py
• tox.ini

Changes:

CONTRIBUTING.rst

---

...	...	@@ -1736,10 +1736,8 @@ obtain profiles::
1736	1736	ForceCommand BST_PROFILE=artifact-receive cd /tmp && bst-artifact-receive --pull-url https://example.com/ /home/artifacts/artifacts
1737	1737
1738	1738
1739		-The MANIFEST.in and setup.py
1740		-----------------------------
1741		-When adding a dependency to BuildStream, it's important to update the setup.py accordingly.
1742		-
	1739	+Managing data files
	1740	+-------------------
1743	1741	When adding data files which need to be discovered at runtime by BuildStream, update setup.py accordingly.
1744	1742
1745	1743	When adding data files for the purpose of docs or tests, or anything that is not covered by
...	...	@@ -1749,3 +1747,22 @@ At any time, running the following command to create a source distribution shoul
1749	1747	creating a tarball which contains everything we want it to include::
1750	1748
1751	1749	./setup.py sdist
	1750	+
	1751	+
	1752	+Updating BuildStream's Python dependencies
	1753	+------------------------------------------
	1754	+BuildStream's Python dependencies are listed in multiple
	1755	+`requirements files <https://pip.readthedocs.io/en/latest/reference/pip_install/#requirements-file-format>`
	1756	+present in the `requirements` directory.
	1757	+
	1758	+All `.txt` files in this directory are generated from the corresponding `.in`
	1759	+file, and each `.in` file corresponds to a certain kind of dependencies. For
	1760	+example, `requirements.in` contains all runtime dependencies of BuildStream.
	1761	+`requirements.txt` is generated from it, and contains pinned versions of all
	1762	+runtime dependencies (including transitive) of BuildStream.
	1763	+
	1764	+When adding a new dependency to BuildStream, or updating existing dependencies,
	1765	+it is important to update the appropriate requirements file accordingly. After
	1766	+changing the `.in` file, run the following to update the matching `.txt` file::
	1767	+
	1768	+ make -C requirements

MANIFEST.in

---

...	...	@@ -32,12 +32,12 @@ include .pylintrc
32	32	recursive-include buildstream/_protos *.proto
33	33
34	34	# Requirements files
35		-include tools/requirements.in
36		-include tools/requirements.txt
37		-include tools/dev-requirements.in
38		-include tools/dev-requirements.txt
39		-include tools/plugin-requirements.in
40		-include tools/plugin-requirements.txt
	35	+include requirements/requirements.in
	36	+include requirements/requirements.txt
	37	+include requirements/dev-requirements.in
	38	+include requirements/dev-requirements.txt
	39	+include requirements/plugin-requirements.in
	40	+include requirements/plugin-requirements.txt
41	41
42	42	# Versioneer
43	43	include versioneer.py

buildstream/_frontend/complete.py

---

...	...	@@ -31,7 +31,7 @@
31	31	# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
32	32	# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
33	33	#
34		-import collections
	34	+import collections.abc
35	35	import copy
36	36	import os
37	37
...	...	@@ -218,7 +218,7 @@ def is_incomplete_argument(current_params, cmd_param):
218	218	return True
219	219	if cmd_param.nargs == -1:
220	220	return True
221		- if isinstance(current_param_values, collections.Iterable) \
	221	+ if isinstance(current_param_values, collections.abc.Iterable) \
222	222	and cmd_param.nargs > 1 and len(current_param_values) < cmd_param.nargs:
223	223	return True
224	224	return False

buildstream/_yaml.py

---

...	...	@@ -287,7 +287,7 @@ def node_decorate_dict(filename, target, source, toplevel):
287	287	provenance.members[key] = member
288	288
289	289	target_value = target.get(key)
290		- if isinstance(value, collections.Mapping):
	290	+ if isinstance(value, collections.abc.Mapping):
291	291	node_decorate_dict(filename, target_value, value, toplevel)
292	292	elif isinstance(value, list):
293	293	member.elements = node_decorate_list(filename, target_value, value, toplevel)
...	...	@@ -302,7 +302,7 @@ def node_decorate_list(filename, target, source, toplevel):
302	302	target_item = target[idx]
303	303	element = ElementProvenance(filename, source, idx, toplevel)
304	304
305		- if isinstance(item, collections.Mapping):
	305	+ if isinstance(item, collections.abc.Mapping):
306	306	node_decorate_dict(filename, target_item, item, toplevel)
307	307	elif isinstance(item, list):
308	308	element.elements = node_decorate_list(filename, target_item, item, toplevel)
...	...	@@ -578,7 +578,7 @@ def is_ruamel_str(value):
578	578	#
579	579	def is_composite_list(node):
580	580
581		- if isinstance(node, collections.Mapping):
	581	+ if isinstance(node, collections.abc.Mapping):
582	582	has_directives = False
583	583	has_keys = False
584	584
...	...	@@ -847,7 +847,7 @@ def composite_dict(target, source, path=None):
847	847
848	848	target_value = target.get(key)
849	849
850		- if isinstance(source_value, collections.Mapping):
	850	+ if isinstance(source_value, collections.abc.Mapping):
851	851
852	852	# Handle creating new dicts on target side
853	853	if target_value is None:
...	...	@@ -862,7 +862,7 @@ def composite_dict(target, source, path=None):
862	862	# Add a new provenance member element to the containing dict
863	863	target_provenance.members[key] = source_provenance.members[key]
864	864
865		- if not isinstance(target_value, collections.Mapping):
	865	+ if not isinstance(target_value, collections.abc.Mapping):
866	866	raise CompositeTypeError(thispath, type(target_value), type(source_value))
867	867
868	868	# Recurse into matching dictionary
...	...	@@ -923,7 +923,7 @@ RoundTripRepresenter.add_representer(SanitizedDict,
923	923	#
924	924	def node_sanitize(node):
925	925
926		- if isinstance(node, collections.Mapping):
	926	+ if isinstance(node, collections.abc.Mapping):
927	927
928	928	result = SanitizedDict()
929	929
...	...	@@ -1067,7 +1067,7 @@ class ChainMap(collections.ChainMap):
1067	1067	def node_chain_copy(source):
1068	1068	copy = ChainMap({}, source)
1069	1069	for key, value in source.items():
1070		- if isinstance(value, collections.Mapping):
	1070	+ if isinstance(value, collections.abc.Mapping):
1071	1071	copy[key] = node_chain_copy(value)
1072	1072	elif isinstance(value, list):
1073	1073	copy[key] = list_chain_copy(value)
...	...	@@ -1080,7 +1080,7 @@ def node_chain_copy(source):
1080	1080	def list_chain_copy(source):
1081	1081	copy = []
1082	1082	for item in source:
1083		- if isinstance(item, collections.Mapping):
	1083	+ if isinstance(item, collections.abc.Mapping):
1084	1084	copy.append(node_chain_copy(item))
1085	1085	elif isinstance(item, list):
1086	1086	copy.append(list_chain_copy(item))
...	...	@@ -1095,7 +1095,7 @@ def list_chain_copy(source):
1095	1095	def node_copy(source):
1096	1096	copy = {}
1097	1097	for key, value in source.items():
1098		- if isinstance(value, collections.Mapping):
	1098	+ if isinstance(value, collections.abc.Mapping):
1099	1099	copy[key] = node_copy(value)
1100	1100	elif isinstance(value, list):
1101	1101	copy[key] = list_copy(value)
...	...	@@ -1112,7 +1112,7 @@ def node_copy(source):
1112	1112	def list_copy(source):
1113	1113	copy = []
1114	1114	for item in source:
1115		- if isinstance(item, collections.Mapping):
	1115	+ if isinstance(item, collections.abc.Mapping):
1116	1116	copy.append(node_copy(item))
1117	1117	elif isinstance(item, list):
1118	1118	copy.append(list_copy(item))
...	...	@@ -1147,7 +1147,7 @@ def node_final_assertions(node):
1147	1147	raise LoadError(LoadErrorReason.TRAILING_LIST_DIRECTIVE,
1148	1148	"{}: Attempt to override non-existing list".format(provenance))
1149	1149
1150		- if isinstance(value, collections.Mapping):
	1150	+ if isinstance(value, collections.abc.Mapping):
1151	1151	node_final_assertions(value)
1152	1152	elif isinstance(value, list):
1153	1153	list_final_assertions(value)
...	...	@@ -1155,7 +1155,7 @@ def node_final_assertions(node):
1155	1155
1156	1156	def list_final_assertions(values):
1157	1157	for value in values:
1158		- if isinstance(value, collections.Mapping):
	1158	+ if isinstance(value, collections.abc.Mapping):
1159	1159	node_final_assertions(value)
1160	1160	elif isinstance(value, list):
1161	1161	list_final_assertions(value)

requirements/Makefile

---

	1	+# Makefile for updating BuildStream's requirements files.
	2	+#
	3	+
	4	+REQUIREMENTS_IN := $(wildcard *.in)
	5	+PYTHON := python3
	6	+VENV := $(PYTHON) -m venv
	7	+
	8	+REQUIREMENTS_TXT = $(REQUIREMENTS_IN:.in=.txt)
	9	+VENV_PIP = $(VENVDIR)/bin/pip
	10	+
	11	+
	12	+.PHONY: all
	13	+
	14	+all: $(REQUIREMENTS_TXT)
	15	+
	16	+%.txt: %.in
	17	+ $(eval VENVDIR := $(shell mktemp -d $$TMPDIR/bst-venv.XXXXXX))
	18	+ $(VENV) $(VENVDIR)
	19	+ $(VENV_PIP) install -r $^
	20	+ $(VENV_PIP) freeze -r $^ > $@
	21	+ rm -rf $(VENVDIR)

tools/dev-requirements.in → requirements/dev-requirements.in

---

tools/dev-requirements.txt → requirements/dev-requirements.txt

---

tools/plugin-requirements.in → requirements/plugin-requirements.in

---

tools/plugin-requirements.txt → requirements/plugin-requirements.txt

---

tools/requirements.in → requirements/requirements.in

---

tools/requirements.txt → requirements/requirements.txt

---

setup.py

---

...	...	@@ -270,10 +270,10 @@ def get_cmdclass():
270	270	#####################################################
271	271	# Gather requirements #
272	272	#####################################################
273		-with open('tools/dev-requirements.in') as dev_reqs:
	273	+with open('requirements/dev-requirements.in') as dev_reqs:
274	274	dev_requires = dev_reqs.read().splitlines()
275	275
276		-with open('tools/requirements.in') as install_reqs:
	276	+with open('requirements/requirements.in') as install_reqs:
277	277	install_requires = install_reqs.read().splitlines()
278	278
279	279	#####################################################

tox.ini

---

...	...	@@ -5,9 +5,9 @@ skip_missing_interpreters = true
5	5	[testenv]
6	6	commands = pytest {posargs}
7	7	deps =
8		- -rtools/requirements.txt
9		- -rtools/dev-requirements.txt
10		- -rtools/plugin-requirements.txt
	8	+ -rrequirements/requirements.txt
	9	+ -rrequirements/dev-requirements.txt
	10	+ -rrequirements/plugin-requirements.txt
11	11	passenv =
12	12	BST_FORCE_BACKEND
13	13	GI_TYPELIB_PATH
...	...	@@ -18,9 +18,9 @@ commands =
18	18	pycodestyle
19	19	pylint buildstream
20	20	deps =
21		- -rtools/requirements.txt
22		- -rtools/dev-requirements.txt
23		- -rtools/plugin-requirements.txt
	21	+ -rrequirements/requirements.txt
	22	+ -rrequirements/dev-requirements.txt
	23	+ -rrequirements/plugin-requirements.txt
24	24
25	25	[testenv:docs]
26	26	commands =
...	...	@@ -30,8 +30,8 @@ deps =
30	30	sphinx==1.7.9
31	31	sphinx-click
32	32	sphinx_rtd_theme
33		- -rtools/requirements.txt
34		- -rtools/plugin-requirements.txt
	33	+ -rrequirements/requirements.txt
	34	+ -rrequirements/plugin-requirements.txt
35	35	passenv =
36	36	BST_FORCE_SESSION_REBUILD
37	37	BST_SOURCE_CACHE

—
View it on GitLab.
You're receiving this email because of your account on gitlab.com. If you'd like to receive fewer emails, you can adjust your notification settings.