Re: survey Friends of GNOME donors

[Stormy Peters <stormy gnome org>]

FYI, Behdad pointed me at LimeSurvey which he had installed and I created a draft, http://www.gnome.org/~behdad/survey/index.php?sid=16424&lang=en.

On Thu, May 21, 2009 at 3:53 PM, John Williams <john williams lists gmail com> wrote:

I teach marketing research, so I feel able to comment with a fair degree of background knowledge.  

Awesome! Glad to have someone who knows something about surveys.

 

Number one survey tip: don't include ANY question unless you really need it. 

I'd agree with that.

Example: Why on Earth would you want to know about the respondents demographic profile?  If you want to "get inside their head" ask them about their opinions, beliefs and feelings.

After creating the questions, I'm inclined to agree with you, at least on income and education.

I would like to know the gender breakdown because I think Friends of GNOME might be more users than just developers and I'd like to know if women are using GNOME. They aren't developing it in large numbers. I think encouraging women to participate is something the GNOME community has done and wants to continue.

I'd also like to know the breakdown of students, developers, retired folks, etc as I think it will help us recruit in the future. If 50% of respondents are students, then we should be working more closely with universities.

2. Use survey monkey

As a last resort.  Better to use an on-line survey tool of your own, that way you have more control.  Don't let survey monkey do the analysis for you.  Get the data from them and do the analysis yourself.

So we've got LimeSurvey, an open source tool now ...

3. No open ended questions. Use choices with an other option and room for comments.

Complete horse-shit.  Open-ended questions are best (they are information-rich), but they are time-consuming to analyse. Only use closed questions when you are _absolutely_ sure that you have a complete and mutually exclusive list of all possible responses.  That's often harder than it seems.

I added an other to every option.  And there are several open ended questions. But not all of them are.

5. Less than 3 minutes. "Please help us by taking this survey in 3 minutes or less."

Nice.  Except it's better worded as something like

"We'd like to ask for your help in finding out how we can do better.  When we tested this survey we found that most people could complete it in three minutes or less.  If you can help us by participating, we will use the information to improve our offerings."

I think that would go well in the email asking them to participate, but at the top of the survey shouldn't it be shorter?

Goal: I think the most important thing to find out is "why GNOME?" Why did they decide to support us? (Are they users, developers, ... How did they find out about donating? etc.)

Once you have that information, what will you _do_ with it?  How will it inform and guide management _action_? 

That depends on each question. Some questions will help us know where to recruit developers, others where to advertise Friends of GNOME, others what applications people value, ...

 

You donated to Friends of GNOME in the past year. Please help us by taking this survey in 3 minutes or less.

1. Are you ... (check all that apply)
- GNOME user
- GNOME developer
- GNOME contributor (event planning, documentation, translation, web page creation, ...)
- GNOME Foundation member
- GNOME fan
- Other ______________

(Some further explanation --- by popup, hover etc. is helpful. For example: If I develop on GNOME but don't have a module in the official module-set, am I a GNOME developer?

There's a help option for every question. If people want to suggest text or at least which ones are confusing, I'll add it.

2. Do you use ... (check all that apply)
- GNOME desktop
- GNOME applications like Abiword, Gimp, Banshee, Inkscape (which ones should we list here?)
- GNOME products on Windows
- GNOME products on KDE
- None of the above

What is the purpose of this question?  I can't see it.

We don't have a clue what people use or what they value. We assume that everyone using GNOME is using the "GNOME desktop" but we don't know that. What if all they are using is GNOME games on the default distro that comes with their Eee PC?

We'll learn what's of value and where to invest our resources (both development and marketing.)

 

3. Which GNOME Foundation activities do you value most?
- Hackfests
- GUADEC
- 6 month releases
- web pages
- forums
- translations
- documentation
- other ____________

Nice.  Perhaps some more on what you mean by "value"?

How about "Which GNOME Foundation activities do you think are most important?"

 

4. What would you like to see the GNOME Foundation do over the next year? (check all that apply)
- exactly what it's doing now
- more developer events
- more outreach to end users
- more end user events
- more new products
- more support of existing products
- more work with distributors

Excellent.  But is this list exhaustive?

No. Feel free to suggest more!

 

5. How did you hear about the Friends of GNOME donation page?
- from the gnome.org web page
- from Planet GNOME
- from a friend
- from a blog
- from an email
- other _________

What is the purpose of this question?

To know which avenues of outreach have been most successful so we can do those more and the others less.

What is your annual household income? [Are there standards for this?]
- $0-$20,000
- $20,000 - $40,000
- $40,000 - $60,000
- $60,000 - $80,000
- $80,000 - $100,000
- $100,000 - $149,000
- $150,000+

I don't see the relevance of this question.

Me neither. I'm quite happy to remove it. :)

What is your current education level?
- some school
- high school equivalent
- some college
- college degree
- graduate degree
- other___________

Or this one.

Same here.

Thanks for kicking this off, Stormy.  I have tried to make my critique constructive; I hope it doesn't read too negatively ;-)

The feedback is great!

P.S.  I'm teaching an Honours course in Marketing Research in the second half of this year (it's near the end of the first semester here in New Zealand).  There is a chance that we could make your problem part of the course work.  Would that be helpful?  The problem from your point of view would be that you wouldn't get a final report from students until October or so.

That's an awesome offer!

I'd like to do this sooner as we are doing a big Friends push right now. However, we always need round #2 or we could survey a different group of people.

The other people I'd really like to survey is GNOME Foundation members. I'd like to find out what the Foundation could do for them.

And the group I'd really, really like to survey is GNOME users but I don't have a great way to reach them except to assume that Friends of GNOME, GNOME Foundation members, etc are all GNOME users.

Best,

Stormy