Hello,As before, a notebook containing *all* of the results should be available via following link:
https://mybinder.org/v2/gh/james-ennis/bst-benchmarks-notebooks/master?filepath=all_results.ipynbIf the link has expired, you'll need to go to https://mybinder.org/ and the copy and paste: https://github.com/james-ennis/bst-benchmarks-notebooks into the url field, with branch "master" and file "all_results.ipynb". Then "launch".
When the notebook has loaded please click on "Kernel" -> "Restart and Run all".
This will show the four interactive graphs and some instructions regarding how the toolbar can be used to interact with each graph.
-----All results were obtained on the same hardware (a Codethink dev machine configured to be a gitlab runner - not being used for anything else), and the results files are being appended to by a CI job. Specs as follows:
* Linux (Debian stable) * x86_64 * 16G RAM * 500Gb SSD * Intel i7-3770 * 8 cores @ 3.40 GHz -----Here are the times and max-rss (peak memory usage) results for a `bst show` of base-files/base-files.bst in the Debian-like project [0] (jennis/use_remote_file branch [1]) for the last 5 weeks:
┌─────┬─────────────────┬─────────────────┬─────────────────┐ │ │ Show │ YAML cache show │ Show once built │ ├─────┼───────┬─────────┼───────┬─────────┼───────┬─────────┤ │Dates│Time │Memory │Time │Memory │Time │Memory │ │ │(s) │(M) │(s) │(M) │(s) │(M) │ ├─────┼───────┼─────────┼───────┼─────────┼───────┼─────────┤ │04/15│11.29 │351 │11.23 │352 │12.97 │470 │ │04/21│ │ │ │ │ │ │ ├─────┼───────┼─────────┼───────┼─────────┼───────┼─────────┤ │04/22│11.50 │259 │11.44 │259 │13.21 │314 │ │04/28│ │ │ │ │ │ │ ├─────┼───────┼─────────┼───────┼─────────┼───────┼─────────┤ │04/29│11.78 │186 │11.58 │186 │13.28 │194 │ │05/05│ │ │ │ │ │ │ ├─────┼───────┼─────────┼───────┼─────────┼───────┼─────────┤ │05/06│11.91 │185 │11.58 │186 │13.94 │193 │ │05/12│ │ │ │ │ │ │ ├─────┼───────┼─────────┼───────┼─────────┼───────┼─────────┤ │05/13│11.65 │186 │11.50 │186 │12.88 │200 │ │05/19│ │ │ │ │ │ │ └─────┴───────┴─────────┴───────┴─────────┴───────┴─────────┘ -----Here are the times and max-rss results for a `bst build` (4, 8 and 12 builders) of base-files/base-files.bst in the Debian-like project [1].
┌─────┬─────────────────┬─────────────────┬──────────────────┐ │ │ 4 Builders │ 8 Builders │ 12 Builders │ ├─────┼───────┬─────────┼───────┬─────────┼───────┬──────────┤ │Date │Time │Memory │Time │Memory │Time │Memory │ │ │(s) │(M) │(s) │(M) │(s) │(M) │ ├─────┼───────┼─────────┼───────┼─────────┼───────┼──────────┤ │04/08│233.83 │354 │249.45 │354 │267.39 │354 │ │04/14│ │ │ │ │ │ │ ├─────┼───────┼─────────┼───────┼─────────┼───────┼──────────┤ │04/22│187.29 │262 │207.31 │262 │226.77 │262 │ │04/28│ │ │ │ │ │ │ ├─────┼───────┼─────────┼───────┼─────────┼───────┼──────────┤ │04/29│165.02 │190 │182.59 │190 │207.50 │190 │ │05/05│ │ │ │ │ │ │ ├─────┼───────┼─────────┼───────┼─────────┼───────┼──────────│ │05/06│176.05 │190 │196.46 │190 │219.28 │190 │ │05/12│ │ │ │ │ │ │ ├─────┼───────┼─────────┼───────┼─────────┼───────┼──────────┤ │05/13│168.47 │197 │189.70 │198 │213.01 │197 │ │05/19│ │ │ │ │ │ │ └─────┴───────┴─────────┴───────┴─────────┴───────┴──────────┘ Please see [2] if you're unable to open the weekly notebook.The in time improvements to show and build have mainly come from the landing of raoul/974-Artifact-Rework (!1292 [3]), where we are now seeing (on the graphs) that the time to show *built* elements is quicker than the initial show! Also, our build times have all decreased as well. Unfortunately, this also increased our memory usage but only by a little, so I do not think we should be too concerned about this.
Thanks, James [0] https://gitlab.com/jennis/debian-stretch-bst[1] https://gitlab.com/jennis/debian-stretch-bst/tree/jennis/use_remote_file [2] https://mail.gnome.org/archives/buildstream-list/2019-February/msg00046.html
[3] https://gitlab.com/BuildStream/buildstream/merge_requests/1292
Attachment:
weekly_results.ipynb
Description: Text document