Re: [Usability] user levels, etc.

[Maciej Stachowiak <mjs noisehavoc org>]

On 12Nov2001 09:51PM (-0500), Luis Villa wrote:
> On Mon, 2001-11-12 at 21:39, Adam Elman wrote:
> > The truth is, however, that in usability testing it's usually possible 
> > to get very good results with a very small number of people.  There is a 
> > lot of usability literature that indicates that you tend to get most of 
> > the same results for a reasonably well-chosen group of, say, 3-5 people 
> > as for a much larger group.  This is not, of course, true of psychology 
> > or political polling.
> 
> Fair enough. Not having read the research, I'm not really qualified to
> say anything, but I will say that the skeptic is me is really, really
> curious as to how 'most of the same results' is defined. My gut feeling
> is that if you are aiming for knowledge about 75% of users, 3-5 would be
> fine, but we shouldn't settle for that. We should aim to do much better
> than that.
>  

Usability being a pragmatic field, one often finds that 10% of the
problem areas cause 90% of the problems experienced by users. That's
part of why a relatively small sample will tend to find nearly all of
the most important problems, rather than the 75% figure you guessed.

Regards,

Maciej