Thin Client Benchmark Testing

[Dave Richards <drichard largo com>]

(( resend ))

All-
      There were some conversations on the IRC about GTK performance on our thin clients.  I wanted to build a baseline for myself to start with so I have something to measure as tuning continues.  For our needs, we don't ever use the system console so performance for us is how well GNOME applications remote display to our devices.  

Our most common device is an NCD Explora 451, circa 1996.  Up to GTK 2.6 they have worked fine.  They obviously aren't as fast as new devices, but work well for the GNOME desktop, Evolution, OpenOffice and Firefox.

Our new thin clients are HP devices, running 800Mhz Crusoe chips with a mini-Linux kernel on them.

The third 'device' is Citrix Metaframe For Unix which allows us to deploy Linux applications to remote sites with less than fiber optic speeds, and also to handheld devices and home users.  It's designed to run over low bandwidth and starts up an Xserver on Solaris for Sparc.

The best benchmark would have been to have done all of these tests on the same hardware, but that option wasn't available to me.  The GTK 2.8 server has far more horsepower than the GTK 2.6 machine and yet produced slower results.

GTK 2.6 was a single 2.8Ghz CPU, 100Mb network line.
GTK 2.8 was a quad 3.66Ghz CPU, 1Gb network line.

To create at least a basic measurement of speed, I used gtkperf from Sourceforge.

GTK 2.8 doesn't work on Metaframe at all in 16 and crashed, I opened a bug this morning:

http://bugzilla.gnome.org/show_bug.cgi?id=320399


Here are the results

Test	GTK 2.6 NCD 16 Bit X	GTK 2.6 HP 16 Bit X	GTK 2.6 Citrix 16 Bit X	GTK 2.8 NCD 16 Bit X	GTK 2.8 HP 16 Bit X	GTK 2.8 Citrix 16 Bit X
GtkEntry	4.27	0.43	1.07	5.19	0.54	Crashed
GtkComboBox	88.64	12.71	21.35	140.18	17.42
GtkComboBoxEntry	76.81	12.08	20.50	133.27	17.02
GtkSpinButton	9.58	0.80	2.10	13.28	0.94
GtkProgressBar	8.02	0.88	1.87	8.81	1.04
GtkToggleButton	27.63	4.55	7.49	48.50	4.86
GtkCheckButton	28.07	4.92	7.63	52.10	5.35
GtkRadioButton	36.35	6.94	9.05	66.83	7.01
GtkTextView – Add Text	32.46	4.35	7.80	42.94	3.97
GtkTextView - Scroll	39.78	1.26	9.05	10.02	2.04
GtkDrawingArea - Lines	10.27	0.93	5.34	11.03	0.93
GtkDrawingArea - Circles	19.72	1.84	3.58	19.92	1.85
GtkDrawingArea - Text	115.91	6.43	22.04	82.37	9.07
GtkDrawingArea - Pixbufs	14.39	3.79	3.97	12.85	2.77
Total	511.90	61.92	123.25	647.31	74.80

My random thoughts.
- It seems that programming and design have a lot more to do with speed than hardware.  GTK 2.8 has far more capacity and should have been able to easily smoke the GTK 2.6 tests.
- GTK 2.8 was slower, even on brand new (HP) hardware.
- Are there possible threading or multi-CPU bugs?

I'm now going to start some focused testing on Evolution and see why it's so slow on SL 10.  I'm probably going to lose that machine soon in favor of SL 9.3, because we have to get it ready for production and it looks like SL 10 just isn't going to work in the short term.

Regards
Dave