Re: [Rhythmbox-devel] Usability report on Rhythmbox

[Jonathan Matthew <jonathan d14n org>]

Hi Charline,

On Fri, Jul 23, 2010 at 12:38 AM, Charline
<charline poirier canonical com> wrote:
> Hello,
>
> I have just completed usability testing of Rhythmbox.  I am planning to
> publish the results on CanonicalDesign.com on Tuesday.  I would like you to
> read the report, if you have time, and send  me feedback or any questions
> you might have.
>
> I hope the report will be helpful.

First of all, many thanks for doing this testing. We don't really have
the capacity to do this ourselves, so the only feedback we get is from
users who are motivated and educated enough to report bugs, post to
the mailing list, or talk to us on IRC, which naturally skews towards
more experienced and technically capable users.

The main messages I'm seeing here are that we need to put more effort
into explaining how Rhythmbox interacts with various music sources -
what happens when you import files, what that means when they're on a
removable device; how Rhythmbox treats podcasts and MP3 player devices
and so on; that we need to highlight actions other than playback,
since users are more likely to want to extract an audio CD into their
library than to play from it, and similarly with other devices; and
that we should reduce the time it takes for first time users to make
Rhythmbox do something, by providing shortcuts for importing music
from various locations (somewhere on the filesystem, removable
devices, network shares, etc.). I have some plans for some aspects of
this, but I'm not sure when I'll be able to work on any of it.

My feedback on the report itself:

If UX advocates or developers on the project are part of the intended
audience, it'd be really helpful to clarify the terms you use for
actions and parts of the interface with them ahead of time. I'm having
trouble understanding the descriptions of user actions and
expectations and mapping them to specific parts of the Rhythmbox
interface, and at times it seems like what's being described isn't
part of Rhythmbox at all.

Again, if people who work on the project are part of the intended
audience, starting with 3 and a half pages of unrelenting criticism
isn't a great idea. These are the only people in a position to fix the
problems you're finding, and if you put them on the defensive, they'll
mostly ignore your report, or if you're especially lucky, blame the
conditions of the testing or even the users for everything that went
wrong. I certainly found myself tempted to do that.

Where you make suggestions for simple improvements, or identify
specific missing features, I'd encourage you to file bugs in the
project bug tracker. For example, from this report, your suggestions
about changing the label for the CD extract button and adding
explanations for the 'missing files' and 'import errors' sources could
be cut and pasted straight into a bug report.

I'd like to see more information on how the tasks were presented to
the users. Particularly with the podcast task, it seemed as though
you'd instructed users unfamiliar with podcasts to download one, which
seems to me to be the opposite of how users come to find the podcast
feature in Rhythmbox - they find some media that comes in podcast
form, then search for an application that handles podcasts. In some of
the other tasks, I'm not sure what the user was actually trying to
achieve, which, when also I can't comprehend the descriptions of what
they tried to do, means I'm at a loss to understand how we can improve
the situation.

Again, thanks for doing this, and I hope we can turn this report into
some useful improvements, not just for new users but for everybody.

-jonathan