[Notes] [Git][BuildStream/buildstream][danielsilverstone-ct/use-enumerate] 3 commits: _yaml.py: Do not insert into cache if retrieved from cache

[Daniel Silverstone <gitlab mg gitlab com>]

Title: GitLab

Daniel Silverstone pushed to branch danielsilverstone-ct/use-enumerate at BuildStream / buildstream

Commits:

• 8c41b5b8
  by Daniel Silverstone at 2018-10-26T09:02:30Z

  _yaml.py: Do not insert into cache if retrieved from cache
  
  We were unconditionally inserting into the yaml cache anything we had
  retrieved from the yaml cache.  This had the side-effect of being expensive
  on re-runs of commands where yaml hadn't changed.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

• 494d7018
  by Jonathan Maw at 2018-10-26T09:26:36Z

  Merge branch 'danielsilverstone-ct/do-not-reinsert-yaml' into 'master'
  
  _yaml.py: Do not insert into cache if retrieved from cache
  
  See merge request BuildStream/buildstream!902

• dcc55c46
  by Daniel Silverstone at 2018-10-26T09:31:43Z

  _loader/loadelement.py: Use enumerate() in dependency extraction
  
  In cases where elements end up with large numbers of dependencies,
  the n-squared performance in `_extract_depends_from_node` can produce
  significant delays during project load.  This patch corrects that.
  In addition we were double-retrieving provenance which was unnecessary.
  
  Signed-off-by: Daniel Silverstone <daniel silverstone codethink co uk>
  

2 changed files:

• buildstream/_loader/loadelement.py
• buildstream/_yaml.py

Changes:

buildstream/_loader/loadelement.py

---

...	...	@@ -146,8 +146,8 @@ def _extract_depends_from_node(node, *, key=None):
146	146	depends = _yaml.node_get(node, list, key, default_value=[])
147	147	output_deps = []
148	148
149		- for dep in depends:
150		- dep_provenance = _yaml.node_get_provenance(node, key=key, indices=[depends.index(dep)])
	149	+ for index, dep in enumerate(depends):
	150	+ dep_provenance = _yaml.node_get_provenance(node, key=key, indices=[index])
151	151
152	152	if isinstance(dep, str):
153	153	dependency = Dependency(dep, provenance=dep_provenance, dep_type=default_dep_type)
...	...	@@ -177,10 +177,8 @@ def _extract_depends_from_node(node, *, key=None):
177	177	provenance=dep_provenance)
178	178
179	179	else:
180		- index = depends.index(dep)
181		- p = _yaml.node_get_provenance(node, key=key, indices=[index])
182	180	raise LoadError(LoadErrorReason.INVALID_DATA,
183		- "{}: Dependency is not specified as a string or a dictionary".format(p))
	181	+ "{}: Dependency is not specified as a string or a dictionary".format(dep_provenance))
184	182
185	183	output_deps.append(dependency)
186	184

buildstream/_yaml.py

---

...	...	@@ -204,9 +204,8 @@ def load(filename, shortname=None, copy_tree=False, *, project=None, yaml_cache=
204	204
205	205	if not data:
206	206	data = load_data(contents, file, copy_tree=copy_tree)
207		-
208		- if yaml_cache:
209		- yaml_cache.put_from_key(project, filename, key, data)
	207	+ if yaml_cache:
	208	+ yaml_cache.put_from_key(project, filename, key, data)
210	209
211	210	return data
212	211	except FileNotFoundError as e:

—
View it on GitLab.
You're receiving this email because of your account on gitlab.com. If you'd like to receive fewer emails, you can adjust your notification settings.