Re: quo vadis, docs

[Luis Villa <luis tieguy org>]

2009/2/9 Natan Yellin <aantny gmail com>:
>
>
> On Mon, Feb 9, 2009 at 5:00 PM, Dan Winship <danw gnome org> wrote:
>>
>> Dave Neary wrote:
>> >> - Should we just ditch the docs and declare the UI self-explanatory ?
>> >
>> > Definitely not.
>>
>> Why not? Seems like no one has ever bothered to file bug reports about
>> the fact that they're wrong... Maybe there are as few people reading the
>> docs as there are writing them. In a corporate setting, people will call
>> their help desk when they have problems, and in a home setting, they'll
>> either ask a friend/family member, or ask on a forum. (If people RTFMed
>> first, we wouldn't need an acronym for it.)
>
> This is a moot point unless it can be proven.
>
> If we want to get rid of the docs, we need to run a survey/study first and
> determine how many people read them.

Lack of bug reporting[1] about obviously broken things is the closest
thing we've got to proof, and has in the past contributed to (IMHO)
fairly sound decision making.

Generally, we've never required stricter 'proof' than bug data, for a
couple reasons:

(1) we've got strict (unnecessary?) taboos about data collection from
user machines
(2) implementing valid survey-like data collection is hard.
(3) the alternative, user 'studies', are also hard, time-consuming,
and expensive to do right.
(4) given all of the above, if we required Hard Proof for every design
decision we'd never make any design decisions, because we don't have
any real Hard Proof.

That said:

(1) some other projects are starting to do data-collection-driven UI
studies (Eclipse, OOo, Moz) and perhaps now is the time to start
thinking about doing the same here. We've even got some examples about
how this can be done with very little privacy risk and little
interference in user experience[2] so perhaps the taboos are less
relevant as well.

(2) since bug reporters are almost by definition experts, and docs are
almost by definition useful for non-experts, the lack of bug reports
may be a lot less useful than normal in informing a decision about
docs.

Luis

[1] I'm taking for granted that there are in fact no bug reports; I
really don't know and haven't looked in a long time, though certainly
virtually no reports were filed about docs by regular users when I was
active.

[2] the principles informing the design of http://www.cs.wisc.edu/cbi/
could, IMHO, easily be applied to UI data collection.