Re: [Usability] How to test

[Calum Benson <Calum Benson Sun COM>]

On 30 Oct 2008, at 01:30, Matthew Nuzum wrote:

> Maybe my pencil and paper technique is better for early stage testing
> where there's tons of low-hanging fruit and the videos are better for
> final testing where there is likely to be more contention and debate
> over the possible changes.

It's certainly true that producing videos is time consuming, and they  
aren't necessarily all that useful unless the people responsible for  
the design can't observe the tests first-hand.  (Or unless you need  
some persuasive evidence to show to a manager or some other exec.)

On the other hand, the ability to easily make videos of people using  
their applications-- whether a working prototype, or just a paper  
mockup-- might just encourage people who weren't confident about  
interpreting the results to do the testing anyway and post the videos  
somewhere.  Then the more experienced members of the community could  
help them do the analysis (and for that matter, suggest improvements  
to their testing technique, if appropriate).

When they became more experienced at noticing what's important and  
asking the right questions, then as you say, the need for video is  
reduced.  That said, one other advantage of video is that you don't  
necessarily need to take notes (or at least, as many notes) during the  
test itself, and can concentrate more on the task of facilitation--  
having somebody sitting beside them taking notes does make some  
participants even more nervous than they were to start with, so  
there's a school of thought that it's better to make your notes later  
from the video, if you can.

Cheeri,
Calum.

--
CALUM BENSON, Usability Engineer       Sun Microsystems Ireland
mailto:calum benson sun com            GNOME Desktop Team
http://blogs.sun.com/calum             +353 1 819 9771

Any opinions are personal and not necessarily those of Sun Microsystems