[Gimp-gui] User Testing Report

[<flynn16 tutanota com>]

Hello, GIMP developers:

In January 2018, I conducted a user test on GIMP v2.8, as part of a class project. User testing is the 
process of having representative users from your user personas do a task designed by the interviewer. The 
interviewer then records breakdowns, situations in which the test participant is able to follow a sequence of 
steps toward the final goal, but got stuck at some point, failing to complete the task.

I wrote a lot in the original report, but that was not a good way of communicating. In this email, I will 
summarize the report.

I interviewed 5 people. Two of them are part-time photographers who have 60k+ followers on social media. The 
other three are STEM students who occasionally edit images for class projects. I will name them A, B, C, D, 
E, respectively.

Task: consider the photo of a paper cup stack. Select the cup stack and turn it black-and-white. A rough 
selection is ok.

====================
Issue 1: sub-windows
====================

All test participants expressed their confusion regarding the sub-windows in GIMP v2.8. One of them wondered 
if they belonged to different programs. Despite this, all participants started doing the task without 
changing the default window layout. At the end, I asked each of them to find an existing option that “turns 
everything into one window,” but nobody found it. Participant A looked into the preferences dialog, did not 
find the option, and started browsing all the menubar items. He even opened the “window” menu, but did not 
recognize the option.

> Does the user think of the right thing at the right time (conceptual model)?

No. participant C does not understand the default window layout.
Yes. When instructed, participants want to adjust a program state to accommodate their preferences.

> Is the action visible?

Yes. You can see it in the “window” menu.

> Does the user recognize the action, even if the label is on screen?

No. All test participants fail to discover this action. They expect this action to be in “preferences.”

> Will the user understand the feedback?

Yes, I believe participants will know the action works as they expect, but they prefer to have “single-window 
mode” activated by default.

> Analysis – why does the breakdown happen?

Quoting my textbook [1], default values:
- are the initial state of the program.
- are the best values, reflecting the most common use cases you observe. (Users don’t need to change them.)
Many users don’t change them.

Being completely new to the software, participants did not change the default values. Default values did not 
reflect their expectation.

> Propose a solution!

GIMP should be in single-window mode “by default.”

> Quote your participants!

- “Don’t have two windows. Use one window for the whole program.”
- “When you open the software, you want to focus on the software. You don’t want to see gaps everywhere.”

================================
Issue 2: accepting invalid input
================================

When finding the pen tool, participants D and E first looked at the brush and texture panel, though it is not 
required for the task. In GIMP, it is important to define a selection before using the brush and texture 
panel. The panel is active when a path is in the process of being drawn. A participant interacted with the 
panel without a well-defined region, but this action did not change anything in the image, or change the 
property of the tool.

> Does the user think of the right thing at the right time?

No. The panel lacks a clear title. The participant thinks the panel is some kind of filter.

> Does the user understand the feedback?

No. The panel shows an option is selected, but the option does not make sense in the current mode. The user 
expects a grayed-out panel.

> Analysis – why does the breakdown happen?

No signifier: the panel (or tab) lacks a title.
No constraint: GIMP accepts input that does not make sense in the current mode.

> Propose a solution!

Give each tab a visible title, in addition to the hover text. You can say: brush, texture, gradient.
Gray out options that do not make sense in the current mode. When a path is still being drawn, gray out the 
brush, texture and gradient panels.

==================================
Issue 3: deviation from convention
==================================

* By convention, I mean the previous knowledge which the user has acquired by using other similar software.

Participant D could not close the path to form an area. She clicked on the first vertex, but the path did not 
close. She then dragged the vertices, hoping to make them “stick” together, but the path did not close. 
Getting stuck, she was given a hint to right-click. Following the hint, she converted the path to a closed 
region.

> Does the user think of the right thing at the right time?

Yes. She wants to turn a closed loop into a region.

> Is the action visible?

Yes. The action to make a closed region is in the “tool options.”

> Does the user recognize the action, even if the label is on screen?

No. The participant tried to close the region by double-clicking and dragging the vertices, later got stuck 
and asked for a hint.

> Does the user understand the feedback?

Yes.

> Analysis – why does the breakdown happen?

The participant uses what she learned somewhere else. The action of clicking on the first vertex, or bringing 
two vertices closer, can be used to close a path in other programs, such as Inkscape and Photoshop.

> Propose a solution!

- Show a “four-side arrow” when the cursor hovers on a vertex.
- Leverage what the user has previously learned! Implement a double-click handler that closes the loop, since 
dragging on an existing vertex moves it.
- Make two vertices “stick” together when they are brought close.

=================================
Issue 4: lacking error prevention
=================================

Participant A makes high-quality selection, but GIMP removes his region when he clicks on tool to see what it 
says.

> Propose a solution!

- When appropriate, show a confirmation dialog before deleting the user’s selection.
- When another tool kicks in, delete the selection only after the user clicks on the canvas with the tool.

> Quote your participants!

(Pre-interview) “When I have 300 photos, I don’t have 300 hours to edit them. I apply filters and move on. 
Sometimes I use pre-recorded settings.”

==================================================
Issue 5: non-descriptive names and lack of presets
==================================================

Participant C cannot tell the difference between “invert” and “value invert,” until he reads the tooltip 
text. He cannot understand what “colorize” and “colorify” mean until he opens the dialog.

“Colorize” controls the hue, saturation and brightness. He could have achieve the black-and-white effect by 
turning down the saturation, but he adjusted the hue first, and he was not satisfied with the result. He 
expanded the dropdown for presets, but there wasn’t any preset.

> Propose a solution!

Rephrase the options. For value invert, how about “brightness invert,” as said in the tooltip description.
Value invert ==> Brightness invert
Colorize     ==> Hue, Saturation, Brightness
Colorify     ==> Colored-glass

Don’t leave out the presets. You can start with making simple presets, like “desaturate”.

==========
References
==========

[1] Ko, A. (n.d.). “How to design user interfaces” 
<https://faculty.washington.edu/ajko/books/design-methods/how-to-design-user-interfaces.html 
<https://faculty.washington.edu/ajko/books/design-methods/how-to-design-user-interfaces.html>>


Best regards,

  Flynn Liu