Re: GNOME user survey 2011 (v4)



On Fri, 2011-08-19 at 16:08 +0300, Felipe Contreras wrote:
> >>> Likewise,
> >>> 'happy' will be thought of differently by different people (a very odd
> >>> word to include in a questionnaire, if you don't mind me saying):
> >>
> >> I think everyone understands the word happy.
> >
> > /ME wipes a mouthful of coffee from my monitor
> >
> > Then you haven't read enough of the survey research literature.
> 
> That doesn't change the fact that everyone understands the word "happy".
> 

Not necessary. Just to give an example - there is strong cultural
influence how do you respond to simple question 'How are you'. In some
cultures it is impolite to answer better then 'so so' and the normal
answer is somehow along lines 'it could be worst, it could be better'.
On the other hand the correct response in English is usually 'great' or
'fine' (to quote my teacher 'even if your house is burnt and your dog is
terminally ill'). I have been warned to avoid 'standard' 'so so'
response as I will be perceived as either impolite or after some large
disaster because what I really meant was 'great'.

(Somehow less directly related but also illustrates the problem of
tricky words - in my native language friend means what in English is
understood by close friend and many people whom I would call in English
friend I would call in Polish acquaintance. Even though I know the
difference I am less inclined to call people friends as in my mental
model they are described by word 'acquaintance').

Of course this is 'just' cultural bias caused by people not being native
speakers of English. You need to add individual bias. In each case it
adds more and more 'noise' to survey. 

> >>> tl;dr version: your survey results will be misleading.
> >>
> >> No, the results would not be misleading; the *analysis* of the results
> >> might. But different people can analyze them in different ways. The
> >> important thing is to get *some* results.
> >
> > It seems bizarre to suggest that research data is valid irrespective
> > of how it is gathered. If your questionnaire does not provide valid
> > measurements no amount of analysis can compensate.
> 
> You can thrown an analysis saying all this data is crap if that makes
> you happier, but this survey won't eat babies.
> 

I would argue that incorrect data (misinformation) is worst case then no
data at all.

Regards

Attachment: signature.asc
Description: This is a digitally signed message part



[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]