[BuildStream] Follow on from https://mail.gnome.org/archives/buildstream-list/2018-December/msg00042.html

[Lachlan Mackenzie <lachlan mackenzie codethink co uk>]

Hi,

It was brought to my attention that some results from benchmarking might 
be pertinent for the discussion about build trees, as such I have the 
following results:

Benchmarking tests were run for "Build of Baserock stage1-binutils for 
x86_64" using
the gitlab runner that we have setup for nightly runs together with a 
handcrafted
benchmark configuration file (and handcrafted buildstream spur branch).

The results are as follows:

Pre-merge (Commit prior to build tree caching being merged)
===========================================================

Commit: "Merge branch 'Qinusty/275' into master" - Date: 29-05-2018

Buildstream SHA                            Average time across 3 runs
---                                        --------------------------
364706f3282b840d2ff3268ce83904277c8a63e8 - 79.43s

Merge (Commit associated with the addition of tree caching)
=============================================================

Commit: "Adding caching build trees"     - Date: 29-05-2018

Buildstream SHA                            Average time across 3 runs
---                                        --------------------------
e70237a54e4a0d4b5aa0497de6ea6544abc5c2a8 - 92.433s

The logged results can be found at:

   https://gitlab.com/BuildStream/benchmarks/-/jobs/133094238

Summary
=======

The results show an increase of ~13 seconds in build time going from a 
test run against
a commit just prior to the merge of build tree caching and one run with 
it added.

References
==========

The definition of the test can be found in:

  
https://gitlab.com/BuildStream/benchmarks/blob/master/bst_benchmarks/default.benchmark

The branch used to configure the test is as follows:

  
https://gitlab.com/BuildStream/benchmarks/tree/lachlanmackenzie/HandCraftTestForBuildTreeComparison

The buildstream spur branch (used to avoid project breakage) is as 
follows:

  
https://gitlab.com/BuildStream/buildstream/tree/lachlanmackenzie/debug_benchmark_failure

A description of Baserock can be found at:
  http://wiki.baserock.org/overview/

Regards

Lachlan Mackenzie

Codethink Ltd
3rd Floor, Dale House, 35 Dale Street, MANCHESTER, M1 2Hf, United 
Kingdom
Telephone: +44 161 236 5575
http://www.codethink.co.uk/
We respect your privacy.   See https://www.codethink.co.uk/privacy.html