[Notes] [Git][BuildStream/buildstream][juerg/buffer-size] 4 commits: .gitlab-ci.yml: Do not automatically run "test-wsl" until runner problems are sorted out

[Jürg Billeter <gitlab mg gitlab com>]

Title: GitLab

Jürg Billeter pushed to branch juerg/buffer-size at BuildStream / buildstream

Commits:

• a482b008
  by Javier Jardón at 2019-02-11T19:23:02Z

  .gitlab-ci.yml: Do not automatically run "test-wsl" until runner problems are sorted out
  

• 0816b8b1
  by Jürg Billeter at 2019-02-11T21:29:29Z

  Merge branch 'jjardon/allow_fail_WSL' into 'master'
  
  .gitlab-ci.yml: allow "test-wsl" to fail until runner problems are sorted out
  
  See merge request BuildStream/buildstream!1145

• a7aed65a
  by Jürg Billeter at 2019-02-12T05:52:07Z

  utils.py: Increase buffer size in sha256sum()
  
  Increasing buffer size from 4 kB to 64 kB speeds up read() bandwidth by
  factor 4, according to a very simple benchmark.
  

• 8b9e1d24
  by Jürg Billeter at 2019-02-12T05:52:07Z

  _cas/cascache.py: Increase buffer size in add_object()
  
  Increasing buffer size from 4 kB to 64 kB speeds up read() bandwidth by
  factor 4, according to a very simple benchmark.
  

3 changed files:

• .gitlab-ci.yml
• buildstream/_cas/cascache.py
• buildstream/utils.py

Changes:

.gitlab-ci.yml

---

...	...	@@ -166,6 +166,7 @@ tests-wsl:
166	166
167	167	script:
168	168	- "${TEST_COMMAND}"
	169	+ when: manual
169	170
170	171	# Automatically build documentation for every commit, we want to know
171	172	# if building documentation fails even if we're not deploying it.

buildstream/_cas/cascache.py

---

...	...	@@ -35,6 +35,8 @@ from .._exceptions import CASCacheError
35	35
36	36	from .casremote import BlobNotFound, _CASBatchRead, _CASBatchUpdate
37	37
	38	+_BUFFER_SIZE = 65536
	39	+
38	40
39	41	# A CASCache manages a CAS repository as specified in the Remote Execution API.
40	42	#
...	...	@@ -371,7 +373,7 @@ class CASCache():
371	373	with contextlib.ExitStack() as stack:
372	374	if path is not None and link_directly:
373	375	tmp = stack.enter_context(open(path, 'rb'))
374		- for chunk in iter(lambda: tmp.read(4096), b""):
	376	+ for chunk in iter(lambda: tmp.read(_BUFFER_SIZE), b""):
375	377	h.update(chunk)
376	378	else:
377	379	tmp = stack.enter_context(utils._tempnamedfile(dir=self.tmpdir))
...	...	@@ -380,7 +382,7 @@ class CASCache():
380	382
381	383	if path:
382	384	with open(path, 'rb') as f:
383		- for chunk in iter(lambda: f.read(4096), b""):
	385	+ for chunk in iter(lambda: f.read(_BUFFER_SIZE), b""):
384	386	h.update(chunk)
385	387	tmp.write(chunk)
386	388	else:

buildstream/utils.py

---

...	...	@@ -235,7 +235,7 @@ def sha256sum(filename):
235	235	try:
236	236	h = hashlib.sha256()
237	237	with open(filename, "rb") as f:
238		- for chunk in iter(lambda: f.read(4096), b""):
	238	+ for chunk in iter(lambda: f.read(65536), b""):
239	239	h.update(chunk)
240	240
241	241	except OSError as e:

—
View it on GitLab.
You're receiving this email because of your account on gitlab.com. If you'd like to receive fewer emails, you can adjust your notification settings.