Re: GNOME user survey 2011

[Olav Vitters <olav vitters nl>]

(snipping a lot)

On Mon, Aug 01, 2011 at 12:45:11PM +0300, Felipe Contreras wrote:
> On Mon, Aug 1, 2011 at 12:05 PM, Olav Vitters <olav vitters nl> wrote:
> > On Mon, Aug 01, 2011 at 11:00:31AM +0300, Felipe Contreras wrote:
> >> On Mon, Aug 1, 2011 at 12:49 AM, Olav Vitters <olav vitters nl> wrote:
> > That doesn't reflect what I said. As such, the statement that nothing
> > would be done is not accurate. What I'd rather have is something like
> >  "Does GNOME do everything what you want to do?"
> > and then
> >  "not at all", "somewhat", "mostly", "everything"
> >
> > then maybe a small textbox to indicate what it doesn't do... but that
> > would have to be analyzed (summarized) again.
> 
> Yes, but if a significant percentage of the people answer "not at
> all", or "somewhat", then the survey would be a bit wasted, since now
> you have to wait another year to ask in more detail. You could also go
> to the free text and try to make sense of it, but that would be too
> much work to plot in any sensible way.

I don't see how your suggestion is better? 'Configurability' to me is
meaningless as there is not much what we could do with the outcome.

> >> >> If you don't think it's very important, it could go to the end.
> >> >> Besides, if the results are overwhelmingly pointing to "Too few"?
> >> >> Don't you think it would be worth investigating? If the results end up
> >> >> being "Just enough", then no harm done.
> >> >
> >> > The question is suggestive so the results will be biased. It is better
> >> > to ask something like "does GNOME do what you want?" and then have a
> >> > text field where they can specify what GNOME lacks.
> >>
> >> Something like:
> >>
> >> === . In your opinion, which areas in GNOME need improvement? ===
> >> (matrix)
> >>
> >>   Columns: don't need / a little / some / much
> >>  + applications
> >>  + usability
> >>  + documentation
> >>  + configuration
> >>  + localization (translation)
> >
> > The question already suggests that GNOME needs improvement.
> 
> Are you kidding me? Nothing is perfect, everything needs improvement,
> everyone knows that.

I do not agree. Some things are good enough. Maybe not from a developer
standpoint and it is not that I'd not appreciate further development,
but sometimes I am perfectly happy with the current development.

I thought the question had the intend to check if people like the GNOME
version that they are using?

> However, people do pick the option "don't care" and "don't need" in
> Git's survey, if that's truly the case:
> https://git.wiki.kernel.org/index.php/GitSurvey2010#20._In_your_opinion.2C_which_areas_in_Git_need_improvement.3F

You're not really addressing my concern which is that the question is
suggestive. Furthermore, this seems to indicate that even with such a
question, some people already indicate that Git is good enough.

> > I rather have non-suggestive questions.
> 
> The purpose of the question is not figure out if GNOME is perfect or
> not. But to find out if the user could vote for what to improve in
> GNOME, what would that be. IMO the worst would be if most users select
> "don't care" in all of them, because then you would have no guidance
> at all to what should be done. Unless of course you are not looking
> for areas of improvement.

If people are not happy with certain things, then those things need
improvement. An option might be one of those things, but that is
something which needs to be analyzed, not assumed.

I'd be curious to know what they're happy with as well as what users are
not happy with.

Further, I was not thinking about specifically using a survey as a
method to get improvement ideas. The reply from Germán Póo-Caamaño
also nicely explains other means to get such feedback.

> > Further, I think 'applications' is vague.
> > I don't think many people will know which parts are GNOME and which
> > parts are not.
> >
> > e.g. 'How happy are you with GNOME in regards to'
> >
> > (happy is not as suggestive as you might think; though another word
> > might be better)
> 
> I do think it's suggestive. I also think improvement is suggestive,
> but I think the suggestion that GNOME needs improvement is a good one,
> specially if you are looking for areas of improvement.

I rather have a non-suggestive survey so that the outcome will be more
acceptable by the various developers.

Note that I've read various articles written that it is easier for
people to be critical and negative then to praise. Giving praise is
especially difficult once you've been asked to be critical. I've
suggested a positive tone due to that and seeing this effect in
practice.

> >> >> >> === 06. What channel(s) do you use to request help about GNOME (if any)? ===
> >> >> >
> >> >> > With what purpose is this asked? No support is given on GNOME Bugzilla.
> >> >> > Only minimal on gnome-list.
> >> >>
> >> >> It's important to have a two-way communication with the users, don't
> >> >> you think? So it's important to figure out what channels they actually
> >> >> use. In fact, I was thinking to add another question asking if they
> >> >> feel there's lack of communication with the team.
> >> >
> >> > I don't see giving support as:
> >> >  * something which shouldn't be done
> >> >  * two-way communication
> >> >  * a way that contributors communicate amongst each other
> >> >
> >> > However, we don't really give support. It would be nice to do more,
> >> > but.. it is not done that much (bit on IRC, mailing lists, some forums).
> >> > Though I'll say beforehand that I see support as separate from a
> >> > developer task.
> >>
> >> I still don't see any suggestions.
> >
> > My suggestion is to remove the question.
> 
> If people can respond in other ways: configuration => much improvement
> needed. Then I'm fine with that.

Configuration is a meaningless term to me. I don't understand why we not
just ask a direct question "does it do what you want?".

Loads of options has various bad effects. Maintenance is more difficult,
it becomes more confusing, the option might be something similar to 'do
not crash' (meaning: it should be solved differently an no option should
be added), etc. See for instance the following blog article which more
or less explains what I mean:
http://headrush.typepad.com/creating_passionate_users/2005/06/featuritis_vs_t.html

> >> It is important to know how many people actually use bugzilla, and how
> >> many people ask questions in IRC channels, mailing lists, etc.
> >
> > With what purpose? What would GNOME get out of this information?
> 
> Suppose 90% of responders don't use any channel of communication.
> Perhaps you would then arrive to the conclusion that more channels of
> communications are needed, or the current ones need to be advertised
> better.

But I already said that I think we should have support options and
currently have none? So why ask this?

> >> Maybe:
> >> What channel(s) do you use to communicate with the GNOME team (if any)?
> >
> > This doesn't reflect support anymore?
> 
> That is irrelevant. Why are you pushing so hard against this? Are you
> trying to say that if users don't have any means of giving feedback to
> the GNOME team, that is _good_?

I'm not, I'm just checking as I'm getting confused with (in my view)
the various topic changes.

Initially you talk about support, then communication, then feedback.

I've stated that we do not really have any means of support and wondered
why the question is suddenly different.

>From another answer I gather you want some way to have people vote for
improvement ideas which is I guess what you mean with 'feedback' or do
you mean 'survey' as feedback?

Regarding voting: That was discussed on d-d-l a long time ago. In short:
various concerns. Not against improvement, but in practice: We rather
not improve things via internet voting, but more improve things after
investigating what needs improvement based on actually seeing what does
not work and fix it possibly in another way. For that you'd need to
understand why something is asked for.

I'm not against changing things, I am not against a survey.
-- 
Regards,
Olav