Re: GNOME user survey 2011

[Felipe Contreras <felipe contreras gmail com>]

On Mon, Aug 1, 2011 at 2:37 PM, Olav Vitters <olav vitters nl> wrote:
> (snipping a lot)
>
> On Mon, Aug 01, 2011 at 12:45:11PM +0300, Felipe Contreras wrote:
>> On Mon, Aug 1, 2011 at 12:05 PM, Olav Vitters <olav vitters nl> wrote:
>> > On Mon, Aug 01, 2011 at 11:00:31AM +0300, Felipe Contreras wrote:
>> >> On Mon, Aug 1, 2011 at 12:49 AM, Olav Vitters <olav vitters nl> wrote:
>> > That doesn't reflect what I said. As such, the statement that nothing
>> > would be done is not accurate. What I'd rather have is something like
>> >  "Does GNOME do everything what you want to do?"
>> > and then
>> >  "not at all", "somewhat", "mostly", "everything"
>> >
>> > then maybe a small textbox to indicate what it doesn't do... but that
>> > would have to be analyzed (summarized) again.
>>
>> Yes, but if a significant percentage of the people answer "not at
>> all", or "somewhat", then the survey would be a bit wasted, since now
>> you have to wait another year to ask in more detail. You could also go
>> to the free text and try to make sense of it, but that would be too
>> much work to plot in any sensible way.
>
> I don't see how your suggestion is better? 'Configurability' to me is
> meaningless as there is not much what we could do with the outcome.

It's almost as meaningless as "Does GNOME do everything what you want
to do?", except that you would know that it's related to the
configuration, and not usability.

>> >> >> If you don't think it's very important, it could go to the end.
>> >> >> Besides, if the results are overwhelmingly pointing to "Too few"?
>> >> >> Don't you think it would be worth investigating? If the results end up
>> >> >> being "Just enough", then no harm done.
>> >> >
>> >> > The question is suggestive so the results will be biased. It is better
>> >> > to ask something like "does GNOME do what you want?" and then have a
>> >> > text field where they can specify what GNOME lacks.
>> >>
>> >> Something like:
>> >>
>> >> === . In your opinion, which areas in GNOME need improvement? ===
>> >> (matrix)
>> >>
>> >>   Columns: don't need / a little / some / much
>> >>  + applications
>> >>  + usability
>> >>  + documentation
>> >>  + configuration
>> >>  + localization (translation)
>> >
>> > The question already suggests that GNOME needs improvement.
>>
>> Are you kidding me? Nothing is perfect, everything needs improvement,
>> everyone knows that.
>
> I do not agree. Some things are good enough. Maybe not from a developer
> standpoint and it is not that I'd not appreciate further development,
> but sometimes I am perfectly happy with the current development.

Then what's the point of doing any more development? Let's call GNOME
3.0 the pinnacle of desktop environment software and don't release any
more versions.

No, nothing is perfect, everything needs improvements.

> I thought the question had the intend to check if people like the GNOME
> version that they are using?

Yes, but you didn't like the original question, so I changed it to
something similar to what Git does.

>> However, people do pick the option "don't care" and "don't need" in
>> Git's survey, if that's truly the case:
>> https://git.wiki.kernel.org/index.php/GitSurvey2010#20._In_your_opinion.2C_which_areas_in_Git_need_improvement.3F
>
> You're not really addressing my concern which is that the question is
> suggestive. Furthermore, this seems to indicate that even with such a
> question, some people already indicate that Git is good enough.

If it is suggesting, why a lot of people answer "don't care" in the
Git survey. Either being suggestive is a problem, or it's not. You are
saying two things at the same time.

>> > I rather have non-suggestive questions.
>>
>> The purpose of the question is not figure out if GNOME is perfect or
>> not. But to find out if the user could vote for what to improve in
>> GNOME, what would that be. IMO the worst would be if most users select
>> "don't care" in all of them, because then you would have no guidance
>> at all to what should be done. Unless of course you are not looking
>> for areas of improvement.
>
> If people are not happy with certain things, then those things need
> improvement. An option might be one of those things, but that is
> something which needs to be analyzed, not assumed.
>
> I'd be curious to know what they're happy with as well as what users are
> not happy with.

Yes, but we haven't found a neutral language. Happy is biased towards
one side, and improvement is biased towards the other. However, I'd
say "improvement" is less biased, because it is implicit in each and
every human endeavor, whereas happiness is not.

> Further, I was not thinking about specifically using a survey as a
> method to get improvement ideas.

What is wrong with getting improvement ideas?

> The reply from Germán Póo-Caamaño
> also nicely explains other means to get such feedback.

I don't know what you are talking about.

>> > Further, I think 'applications' is vague.
>> > I don't think many people will know which parts are GNOME and which
>> > parts are not.
>> >
>> > e.g. 'How happy are you with GNOME in regards to'
>> >
>> > (happy is not as suggestive as you might think; though another word
>> > might be better)
>>
>> I do think it's suggestive. I also think improvement is suggestive,
>> but I think the suggestion that GNOME needs improvement is a good one,
>> specially if you are looking for areas of improvement.
>
> I rather have a non-suggestive survey so that the outcome will be more
> acceptable by the various developers.

Oh, the survey is non-suggestive, it's only this question that we are
debating about the tone, and so far, I haven't heard of a
non-suggestive way to ask this question.

All I know, is that the Git survey has been using "improvement" for
years, and nobody is dismissing the results survey. People think Git
needs improvements... That's no surprise.

> Note that I've read various articles written that it is easier for
> people to be critical and negative then to praise. Giving praise is
> especially difficult once you've been asked to be critical. I've
> suggested a positive tone due to that and seeing this effect in
> practice.

So you would rather get praise than get ideas for improvement? I
prefer "improvement" if there's no way to convince you, then
"happiness" is fine.

>> >> >> >> === 06. What channel(s) do you use to request help about GNOME (if any)? ===
>> >> >> >
>> >> >> > With what purpose is this asked? No support is given on GNOME Bugzilla.
>> >> >> > Only minimal on gnome-list.
>> >> >>
>> >> >> It's important to have a two-way communication with the users, don't
>> >> >> you think? So it's important to figure out what channels they actually
>> >> >> use. In fact, I was thinking to add another question asking if they
>> >> >> feel there's lack of communication with the team.
>> >> >
>> >> > I don't see giving support as:
>> >> >  * something which shouldn't be done
>> >> >  * two-way communication
>> >> >  * a way that contributors communicate amongst each other
>> >> >
>> >> > However, we don't really give support. It would be nice to do more,
>> >> > but.. it is not done that much (bit on IRC, mailing lists, some forums).
>> >> > Though I'll say beforehand that I see support as separate from a
>> >> > developer task.
>> >>
>> >> I still don't see any suggestions.
>> >
>> > My suggestion is to remove the question.
>>
>> If people can respond in other ways: configuration => much improvement
>> needed. Then I'm fine with that.
>
> Configuration is a meaningless term to me. I don't understand why we not
> just ask a direct question "does it do what you want?".

After thinking about it you are right. I think that question is mixing
both usability and configuration, and thus would be harder to
determine what actions to do (if any). But perhaps that's not too bad.

> Loads of options has various bad effects. Maintenance is more difficult,
> it becomes more confusing, the option might be something similar to 'do
> not crash' (meaning: it should be solved differently an no option should
> be added), etc. See for instance the following blog article which more
> or less explains what I mean:
> http://headrush.typepad.com/creating_passionate_users/2005/06/featuritis_vs_t.html

Yes, yes, GNOME developers have thrown that argument ad nauseam. But
it's not a valid reason to don't add features:

 1. How do you know you are on the happy user peak without any numbers?
 2. Can't this be solved easily by adding an "advanced mode"

If you are in the happy user peak, then the majority of responders
will say "just enough". No?

Moreover, I am perfectly sure there's way to achieve configurability,
*and* keep the code maintenable, but it requires more work. You don't
have to aim for that kind of quality, but at least you would know
where your users stand.

>> >> It is important to know how many people actually use bugzilla, and how
>> >> many people ask questions in IRC channels, mailing lists, etc.
>> >
>> > With what purpose? What would GNOME get out of this information?
>>
>> Suppose 90% of responders don't use any channel of communication.
>> Perhaps you would then arrive to the conclusion that more channels of
>> communications are needed, or the current ones need to be advertised
>> better.
>
> But I already said that I think we should have support options and
> currently have none? So why ask this?

To see how the usage of different channels is distributed? Say
StackOverflow turned out very high for Git, if something similar
happens for GNOME, maybe some GNOME developers would hang around there
more.

>> >> Maybe:
>> >> What channel(s) do you use to communicate with the GNOME team (if any)?
>> >
>> > This doesn't reflect support anymore?
>>
>> That is irrelevant. Why are you pushing so hard against this? Are you
>> trying to say that if users don't have any means of giving feedback to
>> the GNOME team, that is _good_?
>
> I'm not, I'm just checking as I'm getting confused with (in my view)
> the various topic changes.
>
> Initially you talk about support, then communication, then feedback.
>
> I've stated that we do not really have any means of support and wondered
> why the question is suddenly different.
>
> From another answer I gather you want some way to have people vote for
> improvement ideas which is I guess what you mean with 'feedback' or do
> you mean 'survey' as feedback?

I want to know if people feel they can give feedback, and I also want
to know if they feel they can ask questions somewhere.

Again, in the Git survey actual questions are:
24. Have you tried to get help regarding Git from other people?
25. If yes, did you get these problems resolved quickly and to your liking?
26. What channel(s) did you use to request help?
27. Which communication channel(s) do you use?

And the results are pretty interesting.

> Regarding voting: That was discussed on d-d-l a long time ago. In short:
> various concerns. Not against improvement, but in practice: We rather
> not improve things via internet voting, but more improve things after
> investigating what needs improvement based on actually seeing what does
> not work and fix it possibly in another way. For that you'd need to
> understand why something is asked for.

Yes. Nobody is saying you should mindlessly do something because
people vote for it (nobody does that). It's just another input.

> I'm not against changing things, I am not against a survey.

Good. I am not actually suggesting changes, but if users feel they
need communication channels, I would like that to be reflected in the
survey.

Cheers.

-- 
Felipe Contreras