[Notes] [Git][BuildStream/buildstream][bschubert/cleanup-local-state] 2 commits: Cleanup internal Loader cache after loading elements

[Benjamin Schubert <gitlab mg gitlab com>]

Title: GitLab

Benjamin Schubert pushed to branch bschubert/cleanup-local-state at BuildStream / buildstream

Commits:

• 78cb0699
  by Benjamin Schubert at 2019-02-13T11:36:14Z

  Cleanup internal Loader cache after loading elements
  

• 723b390b
  by Benjamin Schubert at 2019-02-13T11:36:14Z

  Don't register exceptions when not running the testsuite
  
  This fix a problem with the garbage collector not being able to
  clean the MetaElements that are loaded.
  
  On small projects this is not a problem, but in bigger projects, this
  can save a few hundred of MBs at runtime
  

4 changed files:

• buildstream/_exceptions.py
• buildstream/_loader/loader.py
• buildstream/_project.py
• buildstream/element.py

Changes:

buildstream/_exceptions.py

---

...	...	@@ -19,6 +19,7 @@
19	19	# Tiago Gomes <tiago gomes codethink co uk>
20	20
21	21	from enum import Enum
	22	+import os
22	23
23	24	# Disable pylint warnings for whole file here:
24	25	# pylint: disable=global-statement
...	...	@@ -50,6 +51,9 @@ def get_last_exception():
50	51	# Used by regression tests
51	52	#
52	53	def get_last_task_error():
	54	+ if 'BST_TEST_SUITE' not in os.environ:
	55	+ raise BstError("Getting the last task error is only supported when running tests")
	56	+
53	57	global _last_task_error_domain
54	58	global _last_task_error_reason
55	59
...	...	@@ -67,11 +71,12 @@ def get_last_task_error():
67	71	# tests about how things failed in a machine readable way
68	72	#
69	73	def set_last_task_error(domain, reason):
70		- global _last_task_error_domain
71		- global _last_task_error_reason
	74	+ if 'BST_TEST_SUITE' in os.environ:
	75	+ global _last_task_error_domain
	76	+ global _last_task_error_reason
72	77
73		- _last_task_error_domain = domain
74		- _last_task_error_reason = reason
	78	+ _last_task_error_domain = domain
	79	+ _last_task_error_reason = reason
75	80
76	81
77	82	class ErrorDomain(Enum):
...	...	@@ -126,7 +131,8 @@ class BstError(Exception):
126	131	self.reason = reason
127	132
128	133	# Hold on to the last raised exception for testing purposes
129		- _last_exception = self
	134	+ if 'BST_TEST_SUITE' in os.environ:
	135	+ _last_exception = self
130	136
131	137
132	138	# PluginError

buildstream/_loader/loader.py

---

...	...	@@ -157,8 +157,19 @@ class Loader():
157	157	#
158	158	ret.append(loader._collect_element(element))
159	159
	160	+ self._clean_caches()
	161	+
160	162	return ret
161	163
	164	+ def _clean_caches(self):
	165	+ for loader in self._loaders.values():
	166	+ # value may be None with nested junctions without overrides
	167	+ if loader is not None:
	168	+ loader._clean_caches()
	169	+
	170	+ self._meta_elements = {}
	171	+ self._elements = {}
	172	+
162	173	# cleanup():
163	174	#
164	175	# Remove temporary checkout directories of subprojects

buildstream/_project.py

---

...	...	@@ -344,6 +344,8 @@ class Project():
344	344	for meta in meta_elements
345	345	]
346	346
	347	+ Element._clear_meta_elements_cache()
	348	+
347	349	# Now warn about any redundant source references which may have
348	350	# been discovered in the resolve() phase.
349	351	redundant_refs = Element._get_redundant_source_refs()

buildstream/element.py

---

...	...	@@ -933,6 +933,10 @@ class Element(Plugin):
933	933
934	934	return element
935	935
	936	+ @classmethod
	937	+ def _clear_meta_elements_cache(cls):
	938	+ cls.__instantiated_elements = {}
	939	+
936	940	# _get_redundant_source_refs()
937	941	#
938	942	# Fetches a list of (Source, ref) tuples of all the Sources

—
View it on GitLab.
You're receiving this email because of your account on gitlab.com. If you'd like to receive fewer emails, you can adjust your notification settings.