GNOME user survey 2011 (v5)

[Felipe Contreras <felipe contreras gmail com>]

Hi,

Here's the next version of the survey. Unfortunately, it seems this is
not going to be blessed by GNOME, and questionpro.com only allows 10
questions in the free version. I haven't found a better free online
survey, and unless somebody offers hosting for this survey, it would
have to be limited.

Now, there has been a misconception that I am not listening to some
feedback, which is not true. From my point of view there have been two
kinds of feedback:

 a) Criticism to improve the survey

All these comments have been addressed in one form or another.
Unfortunately, I am forced to drop some of the suggested questions to
fit the 10 question limit. Further criticism is still welcome.

 b) Criticism without possibility for improvements

Many people in this category have already accepted that they don't
think an online user survey is actually possible, so there's really no
way to address these.


One problem raised was the issue of self-selection bias, of course,
without any suggestions to get rid of it. All online surveys have this
issue, however, my proposal was actively suggest people to push others
to take the survey (those people would not be self-selected), and have
a question to identify them. Then we can ignore the self-selected
people, or have two analyses. Of course, I didn't get any response.

In a private discussion, Matthew Garret pointed out that there's still
the issue that some people might not have friends, and thus there
wouldn't be people to push them to take the survey. While this is
true, this is something not even professionals can overcome, the only
thing possible is to make a guess about the amount of people in this
category, and thus the amount of bias.

What about other kinds of bias? Would the survey be invalidated if we
missed some group of people? No. There's a process called
poststratification that would allow us to make valid guesses. Yes, it
might turn out that in the end we don't have enough statistical power
to make any conclusions, but we won't be able to say that until we get
the results.

Another misconception is that this survey might cause some damage, by
drawing the wrong conclusions. As I already explained before, raw data
is raw data, and it can't cause damage. What can cause damage is to
make decisions based on wrong conclusions, which is why GNOME
developers might want to ignore the conclusions. Or different people
might try to do different post-hoc power analysis to try to get some
consensus on what conclusions are actually valid.

I will continue to listen for suggestions for improvement, but I fear
GNOME developers don't want to continue improving this survey in this
mailing list, so feel free to contact me personally if you are
interested in v6 (if needed), or follow this:

https://gist.github.com/1128166

Cheers.

=== 01. Which of the following best resemble your desktop? ===
(click to see the image)
(in case you are not sure what GNOME is)
[single choice]

 - Windows
   http://origin.arstechnica.com/images/windows7/Peek%20-%20Before.png
 - Mac OS X
   http://www.guidebookgallery.org/pics/gui/desktop/firstrun/macosx103.png
 - GNOME 2
   http://library.gnome.org/misc/release-notes/2.28/figures/gnome-2.28.png.en
 - GNOME 3
   http://gnome3.org/img/overview-big.png
 - Unity
   http://static.arstechnica.com/shell-windows.png
 - KDE
   http://www.linuxnov.com/wp-content/uploads/2011/07/KDE-4-7-desktop.jpg
 - I can't tell

=== 02. Overall, how satisfied are you with GNOME? ===
[single choice]

 * not at all
 * barely
 * halfway
 * mostly
 * completely

=== 03. Does GNOME do what you want? ===
[single choice]

 * not at all
 * barely
 * halfway
 * mostly
 * completely

=== 04. What is the main reason you are taking this survey? ===
[single choice, with other]

 * Somebody is pushing me
 * I want to provide feedback to the project
 * I feel I have to

 * Other

=== 05. How many years of experience do you have using computers? ===
[numeric]

=== 06. How often do you use a terminal/console? ==
[single choice]

 * What is that?
 * When I have no other option
 * I can't live without them
 * Is there anything else?

=== 07. Have you contributed to the GNOME project? ===
[single choice]

 * Yes
 * No

=== 08. Have you contacted the GNOME team? ===
[single choice]

 * Yes, successfully
 * Yes, unsuccessfully
 * No, I don't know how
 * No, never had the need

=== 09. If you could change three things in GNOME, what would they be? ===
[free form]

=== 10. Do you have any comments or suggestions for the GNOME team? ===
[free form]

-- 
Felipe Contreras