Re: Make'em suffer!

["Manu Cornet" <manu cornet gmail com>]

Hi Carl,

Thanks a lot for your work on the torturer!

New version available from the usual tree:

           git+ssh://dev.laptop.org/git/projects/soc-gtk

> I've also attached the 4 changes as a patchset that you should be able
> to apply with git-am if you prefer to do it that way.

Ok, I've applied them all. I have made a few minor changes to the
output format however, considering that:

* If we're torturing only one widget at a time, the column headers
(showing which value corresponds to what) don't show up, so it's hard
to understand what those numbers are.

* We were missing an important bit of information, the number of iterations.

So I changed the output to look like this:

# 3 x 84 iterations -- Single widget torture
# Widget        Boot-create     Boot-map        Boot-expose
Boot-destroy    Expose  Resize
GtkEntry        0,010724        0,726733        5,22661 0,237928
0,193584        3,61815

and :

# 3 x 84 iterations -- Full torture
# Widget        Boot-create     Boot-map        Boot-expose
Boot-destroy    Expose  Resize
GtkEntry        0,020881        1,32942 6,77665 0,484845
0,189728        2,83106
GtkButton       0,030032        1,36692 71,8463 0,499937
0,212937        4,71837
GtkCheckButton  0,039381        1,4068  16,0918 0,516581
0,412131        2,83594
...etc.

If your postprocessing script is set to ignore #-commented lines, this
should still work fine. I've also added a blank line before each
output (each single widget test, or each ful torture) so that it is
easier to read, I think this shouldn't be a problem?

And what do you think about displaying the average time values instead
of the total time values, so that one could directly compare results
obtained with different numbers of iterations?

> * The tests scale widgets such as buttons up to nearly full-screen. Is
>   that a useful thing to test in any scenario?
>
> * Meanwhile, some widgets that normally would be quite large,
>   (scrollable areas for example), are tested at a very small size
>   during the "boot" stage.

One of the goals was to not only test an engine in its usual working
conditions, but also to stretch it to more unusual situations to see
if it is solid enough to handle them well.

Indeed, very large buttons or very small scrollable areas aren't
really interesting nor common (although I don't think the size has a
very big impact on the boot time), but I think this tool will be used
mainly to compare different engines. And I would tend to say that if
two engines have exactly the same speed in common situations, but one
of them is written well enough to handle more general situations
better and faster, well, it seems fair to give it better "marks" :-)
Does this make sense?

Now the question is: can an engine that was written with a more
"general" state of mind (not assuming anything about widget sizes) be
slower in common situations just because of its increased generality?
I don't really know :)

> * Additionally, when the "large" widgets _are_ scaled up, they are not
>   tested with any significant amount of internal content

Right, this should be easy to change, I'll try to add content to
larger widgets that usually have some.

> * Another dynamic test opportunity is missed in the progress bars
>   which are only ever drawn at the 0.0 position.

Ah yes, this makes sense, I need to enhance this as well.

> Anyway, I hope the patches and/or the feedback are useful for you.

They are most useful indeed, thanks a lot :)

Cheers,
Manu