Re: Make'em suffer!

[Federico Mena Quintero <federico ximian com>]

On Wed, 2006-08-16 at 00:00 -0700, Carl Worth wrote:

> The important question comes down to: Can someone use the results of
> this tool to accurately predict the performance of one theme compared
> to another, (or in my case one version of GTK compared to another),
> when applied to real applications rather than the test cases.
> 
> And the answer to that depends on how accurately the test cases match
> what the real applications do.

I guess GtkWidgetProfiler is just part of a larger profiling
infrastructure which doesn't exist yet :)

Originally I wanted to be able to say, "when the user clicks a button, a
new window full of widgets comes up.  How long does it take to come up?"
For example, my beloved File/Open dialog needs to be able to come up in
under 0.3 seconds.

Or you change components in Evolution - you shouldn't see it repaint; it
should appear instantaneous.  "Instantaneous" probably means "it needs
to be fully repainted before the next vertical retrace of the monitor".

I guess a raw benchmark like Manu's torturer is useful for this:

- It lets you know that you can paint a button in 0.00x seconds, and an
entry in 0.00y seconds.

- If you have a window with 3 buttons and 2 entries, it should take
(3*0.00x + 2*0.00y + a bit more for the window).  If that is not in the
ballpark of the actual result you get, you know there's something wrong.

- You can probably predict whether you'll be able to paint a window from
your app in under <vertical-retrace-time>.  If a theme lets you achieve
that, but another theme doesn't, you need to fix the second theme.

- It lets us see regressions.  "With GTK+ 2.x.y and theme foo I was able
to draw 1000 buttons per second; now with 2.x.(y+5) I can only paint 200
buttons.  What's up with that?"

What real applications do varies *a lot*.  You have custom tree models
(and any treeview in your window will take over the timings of pretty
much any other kind of widget!), custom widgets, different content in
those widgets.  Hopefully the GtkWidgetProfiler remains as a piece that
is easy to plug into apps so that you can run them in profiling mode, or
something like that.

  Federico