Re: Automated benchmarks of BuildStream

[Jim MacArthur <jim macarthur codethink co uk>]

On 23/03/18 08:03, Tristan Van Berkom wrote:
> On Fri, 2018-03-23 at 07:49 +0000, Angelos Evripiotis wrote:
> > Hello!
> >
> > > That said, if people want to view this in CSV, your sample output
> > > could
> > > *really* use some column alignment - looking through a CSV file and
> > > attempting to spot an abnormality is something I've done for other
> > > benchmarks in the past, it's not so uncommon, and is near
> > > impossible
> > > without column alignment.
> > If you're talking about layout, I've found the 'column' command to be
> > particularly helpful when viewing csv's, e.g. for the sample output:
> >
> > -----
> > $ cat <<EOF | column -ts,
> > Name,Version,Total Time,Max RSS
> > Startup time,master,0.2866666666666667,21558.666666666668
> > Startup time,1.1.0,0.19666666666666666,21368
> > Startup time,1.0.1,0.45,31501.333333333332
> > EOF
> >
> > Name          Version  Total Time           Max RSS
> > Startup time  master   0.2866666666666667   21558.666666666668
> > Startup time  1.1.0    0.19666666666666666  21368
> > Startup time  1.0.1    0.45                 31501.333333333332
> > -----
> This is exactly what I'm talking about yes.
>
> It would be nice to have if it was generated already columnized, given
> that this is mostly for human readability as well as spread sheet
> imports. But, if that is not practical for the given flavor of CSV (I
> know these formats can be finicky), then it's not a big deal if it is
> not columnized.
>
> Cheers,
>      -Tristan

The goal I have in my mind is to have a directory produced by the GitLab 
CI system which contains various artifacts of the benchmarking process. 
JSON output from both benchmarks and log scraping will be the primary 
form of data, and we'll also be able to produce visual indicators in 
HTML and SVG or PNG, along with nicely-formatted tables, and CSV format 
for pasting into a spreadsheet to do your own ad-hoc analysis.

Jim