Hi Angelos,
Thanks for the reply, I was hoping someone would have ideas about where we could host these.
I've had a mess around with trying to host the weekly_extract notebook on GitLab and Github, unfortunately I've been unable to get GitLab to render this. Works well for GitHub though.
I then tried viewing the notebook with nbviewer (via the notebook which was stored on my github repo), unfortunately we lose the interactivity meaning that if we do it this way we can no longer click on a data point and view the MR/branch :( Do you have any ideas as to how we could keep the interactivity?
Anyway, I agree that the notebook should be easier to 'just view', so I'm going to try and deploy them to gitlab pages, failing that, I'll commit them to (somewhere on) GitHub.
Thanks, James
On Tue, 2019-02-19 at 12:21 +0000, Angelos Evripiotis wrote: Hi James, This is a good start! To make it easier for folks to view the notebook, you could version it in GitLab, it has support for rendering them now. Perhaps folks could also contribute to it :)
You can also make an nbviewer link out of any web-addressable content, e.g. your attachment on the mailing list message: Make sure to save the notebook after running it, so the output is visible in the rendered version. That doesn’t seem to be the case in the attached version.
Cheers! Angelos Hello all,
IMPORTANT NOTE: To view the (text) tables in this email, you'll need at least 83 chars for word
wrapping.
I've recently been working on generating data and preparing results to benchmark our Debian-like
project. I've gathered data for all of the *merge commits* since 01/01/2019 which show the time
taken to show, show with the YAML cache, build and show once built, Debian's base files (base-
files/base-files.bst from the Debian-like project [0]). This produces a pipeline of ~6100 elements.
Attached is:
1. A zip file with 4 results files (containing all results since 1st of Jan), and a Jupyter notebook
which reads these in these files and produces the graphs.
2. A Jupyter notebook (weekly_results.ipynb), which contains embedded data for the last 7 days worth
of merge commits, this will be used to track our improvements week-by-week.
IMPORTANT: To view the notebooks, you'll need to install Jupyter notebook, matplotlib and the python
package mplcursors. I'd recommend the following:
* apt install jupyter-notebook python-matplotlib
- or equivalent
* pip3 install --user mplcursors
To view the notebooks, I'd recommend downloading the notebook and extracting the zip into an empty
directory and executing `jupyter-notebook`, open the link and then 'Cell' -> 'Run All' (note that
you may have to rerun a cell to reproduce the interactive graph, do this with shift+Enter).
You should then see the graphs, these are pretty self explanatory and they have
some cool interactive features, e.g. you can zoom into certain regions with the tools provided in
the toolbar below the graph. Also, by clicking on a data point, you should be able to see the !MR
and the branch of that (merge) commit.
You should see 4 graphs:
1. Time taken to show base-files/base-files.bst (without YAML cache, with YAML cache and with YAML
cache and populated local cache).
2. Max memory to show base-files/base-files.bst(without YAML cache, with YAML cache and with YAML
cache and populated local cache).
3. Time taken to build base-files/base-files.bst.
4. Max memory when building base-files/base-files.bst.
All results were obtained on the same hardware (a Codethink dev machine configured to be a gitlab
runner - this machine is not being used for anything else), and the results files are being appended
to by a CI job [1]. Here are some specs of the Dev machine:
* Linux (Debian stable)
* x86_64
* 16G RAM
* 500Gb SSD
Now, because we have been focusing on performance since the gathering, I've taken an average of the
results, per week, which I will now send out as weekly updates so that we can track our
improvements.
Below, are the averages of the show and build times of base-files/base-files.bst for the last 4
weeks:
| Dates | Show (s) | Show with cache (s) | Build (s) | Show once built (s) |
|-------------|----------|---------------------|-----------|---------------------|
| 21/01-27/01 | 46.69 | 27.40 | 378.58 | 33.65 |
| 28/01-03/02 | 46.36 | 27.35 | 383.83 | 33.49 |
| 04-02-10/02 | 45.72 | 26.53 | 268.70 | 32.68 |
| 11/02-17/02 | 43.07 | 24.04 | 228.94 | 30.13 |
Below, are the averages for the maximum memory usage for the show and build of base-files/base-
files.bst for the last 4 weeks:
| Dates | Show (MB) | Show with cache (MB) | Build (MB) | Show once built (MB) |
|-------------|-----------|----------------------|------------|----------------------|
| 21/01-27/01 | 1084 | 1108 | 1109 | 1110 |
| 28/01-03/02 | 1083 | 1108 | 1109 | 1110 |
| 04-02-10/02 | 556 | 581 | 582 | 582 |
| 11/02-17/02 | 403 | 425 | 425 | 426 |
Hopefully, over the coming weeks/months, we'll be seeing all of these numbers continuously drop!
Note that in the future, I will not be attching the zip with *all* the data, I'll just leave a
(gitlab) link which will let you download the artifacts. However, I will be attaching the
weekly_results notebook.
Please let me know if you have any thoughts/improvements on the benchmarks. I hope you find the data
and results helpful!
Thanks,
James
[0] https://gitlab.com/jennis/debian-stretch-bst
[1] https://gitlab.com/jennis/benchmark_debian
_______________________________________________
BuildStream-list mailing list
BuildStream-list gnome org
https://mail.gnome.org/mailman/listinfo/buildstream-list
|
