Re: 2006 LinuxQuestions.org Members Choice Awards

[John Williams <john williams lists gmail com>]

On Mon, 2007-02-19 at 17:55 -0300, Santiago Roza wrote:
> On 2/19/07, John Williams <john williams lists gmail com> wrote:
> 
> 
> > Around 1,000 is sufficient if all we want to do is compare
> > two or three percentages (e.g. GNOME, KDE, Windows) with a margin of
> > error of around +/- 3% at the 95% confidence level.
> 
> problem is that our market share (and maybe even the entire gnu/linux
> market share) is probably around that 3% margin, so we'd be talking
> about a 100% error margin.
Not true, but I can see how you would get that impression.  The margin
of error of a sample proportion depends on the population proportion.
When you quote a margin of error for a sample, typically you do not know
the population proportion, so you make a calculation based on the worst
case scenario (maximum possible margin of error), which corresponds to a
population proportion of 50%.

If the population proportion is for GNU/Linux users is 3% and we have a
random sample of 1,000 we should get around 30 GNU/Linux user
respondents.  You are right in the sense that if we have that many
respondents, estimating the population proportions of GNOME versus KDE
users is problematic with these numbers.  (I didn't think of that:
d'oh!).

So a total sample size in the low thousands should be fine.  But that's
almost irrelevant to the cost and difficulty of obtaining an accurate
sample frame issues.  The marginal cost of obtaining and processing
responses is very low compared to the fixed cost of implementing the
recruitment of respondents.

> what about (us) giving away pamphlets in the streets of a few dozen
> cities?  sounds a little too simple, but could work... how many times
> were you stopped (in a very similar way) for some kind of survey?
Great idea.  If you want help designing the pamphlet, I an very willing to participate.

> problem is that these kinds of surveys would be biased towards people
> who care about answering them = people who care too much about their
> computers = above-average technically skilled/interested people =
> people who are more likely to run gnu/linux.
Very, very true.  But that is true of every survey of human populations,
whether it be computer use or use of illicit drugs or views on global
warming.  This is a problem, but it can be ameliorated in relatively
simple ways.  (E.g. give them a survey about things that everyone is
interested in, and have two or three questions about OS and DE use in
there as well.)