I figured I would rant a bit about our data collection so far and what it can mean for the way we collect data and which data is collected. What it can mean to improving Banshee through changes and guided testing. I am using the data Gabriel posted to his blog http://gburt.blogspot.com/2010/03/banshee-metrics.html and the data is at:
http://banshee-project.org/~gburt/banshee-usage-stats.txt
Generally it would be useful for the way we display the data if we label the default if any, then display the procentage favoring the default. This would allow us to both guide testing to hit high intensity paths and also to guide us to which defaults we have selected wrong.
For testing I believe it would be useful to get a page up with guides on how to ensure that you test certain key features. Such a test matrix as could be derived from this would be most helpful in ensuring the continued high quality of Banshee.
Banshee/Screen/IsComposited:
Users: 519
427 : True
92 : False
True makes up 82%
First the prediction from these numbers is that True is going to increase over time as we get out of the box support for 3D acceleration on free drivers. The future is now, all 3 major vendors (Intel, Nvidia, ATI) can now run reliably this way at least on a set of configurations using Open Source which thus means it can be deployed by distributions out of the box.
Thus testing should focus on running Banshee in all the major compositors during the development cycle. Do we have bugs with any of them. Testing non-composited environments is easy but can be pushed back for time.
As the true count is very likely to reach a comfortable number in the 90% soon, can we use this in development for flashy features or clever rendering if we make the assumption that we have such features this can give us? Perhaps in a Banshee next-gen frontend?
Banshee/Screen/Width:
Median: 1,400 px
Banshee/Screen/Height:
Median: 900 px
It would probably be more useful here to get counts for specific resolutions so we can spot high density uses. It might also be useful to add a testing trick to the wiki on how to run a session restricted to lower resolutions for those willing to test. This would let us spot how we optimally display the interface depending on size, can we adapt to this so that on high res we will arrange things to take advantage of this and rearrange, lose least important information as we scale down?
Banshee/MusicLibrarySource/TrackCount:
Avg: 5,230.5
Median: 2,855.0
Some very nice hard numbers to optimize for. It might be nice to establish the highest and the lowest as well and if we have many outliers in each ditch.
Banshee/Configuration/player_window/maximized:
Users: 427
218 : True
209 : False
I would actually be very interested in getting at something like if Banshee is minimized for more than say 60% and over 90% of it's runtime. If we have a high number of these users they likely see Banshee as a service they call upon and we might think about defaulting to minimizing (probably with some kind of first run introduction then).
Near equal split, so we can't assume the user is maximized when designing the interface?
Banshee/Configuration/player_engine/replay_gain_enabled:
Users: 427
230 : False
197 : True
We might want to debate the default for this. Seems to increase the experience of music if available, any cons?
Banshee/Configuration/player_window/show_cover_art:
Users: 427
222 : False
205 : True
Banshee/Configuration/player_engine/equalizer_enabled:
Users: 427
313 : False
114 : True
Banshee/Configuration/interface/show_context_pane:
Users: 427
222 : False
205 : True
Banshee/Configuration/library/write_metadata:
Users: 426
303 : True
123 : False
I believe the default is False, strong indication that we should change.
Banshee/Configuration/library/move_on_info_save:
Users: 426
243 : False
183 : True
Some 43% of users enable this, what is the impact on performance for Banshee and the system as a whole? Indication is also that we should ensure both paths get tested during a release cycle.
Banshee/Configuration/library/write_metadata:
Users: 426
303 : True
123 : False
Banshee/Configuration/library/write_rating:
Users: 426
231 : False
195 : True
Same as above, also what is the impact of having all of these enabled. Do we end up constantly waking the disk and abusing power, do we harm system performance?
Banshee/Configuration/plugins.bpm/auto_enabled:
Users: 348
224 : False
124 : True
Env/CultureInfo:
Users: 519
223 : en-US
51 : en-GB
41 : de-DE
35 :
21 : ru-RU
18 : it-IT
14 : fr-FR
12 : en-CA
11 : en-AU
11 : es-ES
9 : pl-PL
8 : pt-BR
6 : es-CL
5 : es-MX
5 : nl-NL
5 : sv-SE
4 : de-AT
4 : en-IE
4 : es-AR
4 : nb-NO
4 : nl-BE
3 : de-CH
3 : uk-UA
3 : zh-CN
2 : da-DK
2 : es-CO
2 : ja-JP
1 : ca-ES
1 : en-NZ
1 : en-PH
1 : es-DO
1 : es-PE
1 : es-UY
1 : fr-CA
1 : fr-CH
1 : hu-HU
Lesson learned, 57% (assumption is that unknown are not C/POSIX and thus US english) of users are using localized versions of banshee and we should cover this in the testing matrix. Also please when adding new high level concepts it would be nice to call out on the mailing list to get some input on the translatable easily and naturally to convey the same message in a range of our most used languages.
Banshee/BuildVendor:
Users: 519
292 : Ubuntu 9.10
51 : Ubuntu 9.10 c519a80
33 : source-tarball
25 : Banshee / openSUSE_11.2
22 : git-checkout
20 : banshee-project.org OSX 10.5+ i386/Intel
16 : Gentoo/banshee/1.5.4
14 : Ubuntu lucid (development branch) c519a80
7 : Debian GNU/Linux unstable (sid)
6 : Ubuntu 9.10 8fef2fb
4 : Foresight Linux
4 : Ubuntu 9.04
4 : Ubuntu 9.10 0328ab7
4 : Ubuntu 9.10 ffc9f0f
3 : Ubuntu 9.10 424a345
3 : Ubuntu 9.10 d885c99
2 : ArchLinux
2 : Banshee:Alpha / openSUSE_11.2
2 : Ubuntu lucid (development branch) 8fef2fb
1 : Fedora12-1.5.4-0.1.fc12.aa
1 : Fedora12-1.5.4-1.fc12
1 : Fedora13-1.5.4-1.fc13
1 : Fedora14-1.5.4-1.fc14
Clearly Ubuntu is important to cover, having it available on launch day with install instructions in the release buzz blog posts would probably gain us testers and new users who get excited. We should probably form a very close relationship with the nice hyperair and beg him, help him, bribe him to grow the Banshee team for Ubuntu, and ensure that the ppas are ready on day of release with packages. Also buying him a beer might not be a bad investment in appriciation.
- David
