Notebook attached... On 2019-03-25 11:24, James Ennis via buildstream-list wrote:
Hello, I've appended a line of results to the tables that I posted to the list last week, and also attached the notebook. See the linked ML post ([0]) if you're struggling to open the notebook (the graphs have some fun interactive features). Results below: Averages of the show and build times of base-files/base-files.bst [1]:| Date |Show (s)| Show with cache (s)| Build (s)| Show once built (s)||------------|--------|--------------------|----------|--------------------|| 21/01-27/01| 46.69 | 27.40 | 378.58 | 33.65 | | 28/01-03/02| 46.36 | 27.35 | 383.83 | 33.49 | | 04-02-10/02| 45.72 | 26.53 | 268.70 | 32.68 | | 11/02-17/02| 43.07 | 24.04 | 228.94 | 30.13 | | 18/02-24/02| 40.87 | 22.35 | 228.72 | 28.56 | | 25/02-03/03| 39.33 | 19.85 | 240.00 | 26.44 | | 04/03-10/03| 38.84 | 19.03 | 238.12 | 25.86 | | 11/03-17/03| 38.00 | 18.31 | 301.37 | 25.29 | | 18/03-24/03| 36.67 | 16.90 | 353.67 | 23.71 |Averages of the maximum memory usage for the show and build of base-files/base-files.bst:| Date |Show (M)| Show with cache (M)| Build (M)| Show once built (M)||------------|--------|--------------------|----------|--------------------|| 21/01-27/01| 1084 | 1108 | 1109 | 1110 | | 28/01-03/02| 1083 | 1108 | 1109 | 1110 | | 04-02-10/02| 556 | 581 | 582 | 582 | | 11/02-17/02| 403 | 425 | 425 | 426 | | 18/02-24/02| 410 | 424 | 425 | 425 | | 25/02-03/03| 493 | 514 | 515 | 619 | | 04/03-10/03| 519 | 542 | 543 | 695 | | 11/03-17/03| 521 | 544 | 545 | 697 | | 18/03-24/03| 524 | 547 | 548 | 696 |Please don't be alarmed by the spike in build times, this occurred since the landing of Source cache [2]. Basically, local source imports are now staged into the the local cache, whereas they were not previously (there was a shortcut for local imports, why bother staging this in the local cache when it exists within the project...) and because our benchmarks were benchmarking a project of 6000 local source imports, we get this spike in the build time. Because of this, I've created a project which uses 'remote' imports (with a file url) [3], and I will back-generate the data for this. Additionally, all of these benchmarks show that our peak memory consumption is creeping back up. I guess this is because we keep making these trade-off decisions in favor of a faster runtime, which, for now, some of us have agreed that this is not a bad thing, just something that people should be aware of. Fortunately, once the !1257 - YAML New World Order [4] lands, we should see a significant drop in memory, again (as the initial benchmarks of this branch show).[0] https://mail.gnome.org/archives/buildstream-list/2019-February/msg00046.html[1] https://gitlab.com/jennis/benchmark_debian [2] https://gitlab.com/BuildStream/buildstream/merge_requests/1124[3] https://gitlab.com/jennis/debian-stretch-bst/tree/jennis/use_remote_file[4] https://gitlab.com/BuildStream/buildstream/merge_requests/1257 _______________________________________________ buildstream-list mailing list buildstream-list gnome org https://mail.gnome.org/mailman/listinfo/buildstream-list
Attachment:
weekly_results.ipynb
Description: Text document