Re: [Gimp-gui] User Testing Report

[Jehan <jehan girinstud io>]

Thanks for the report. It looks interesting and globally well done, 
especially as a class project, though some things are quite approximate. 
It would have been interesting to contact the GIMP developers first to 
get some things straight.

Below are my remarks:

On 2019-03-24 07:59, Flynn Liu via gimp-gui-list wrote:
> Hello, GIMP developers:
>
> In January 2018, I conducted a user test on GIMP v2.8, as part of a
> class project. User testing is the process of having representative
> users from your user personas do a task designed by the interviewer.
> The interviewer then records breakdowns, situations in which the test
> participant is able to follow a sequence of steps toward the final
> goal, but got stuck at some point, failing to complete the task.
>
> I wrote a lot in the original report, but that was not a good way of
> communicating. In this email, I will summarize the report.

Indeed. For a first communication, a full report (with many pages, I 
guess) might have been a bit too much. :-)

> I interviewed 5 people. Two of them are part-time photographers who
> have 60k+ followers on social media. The other three are STEM students
> who occasionally edit images for class projects. I will name them A,
> B, C, D, E, respectively.
>
> Task: consider the photo of a paper cup stack. Select the cup stack
> and turn it black-and-white. A rough selection is ok.
>
> ====================
> Issue 1: sub-windows
> ====================
>
> All test participants expressed their confusion regarding the
> sub-windows in GIMP v2.8. One of them wondered if they belonged to
> different programs. Despite this, all participants started doing the
> task without changing the default window layout. At the end, I asked
> each of them to find an existing option that “turns everything into
> one window,” but nobody found it. Participant A looked into the
> preferences dialog, did not find the option, and started browsing all
> the menubar items. He even opened the “window” menu, but did not
> recognize the option.
>
> > Does the user think of the right thing at the right time (conceptual 
> > model)?
>
> No. participant C does not understand the default window layout.
> Yes. When instructed, participants want to adjust a program state to
> accommodate their preferences.
>
> > Is the action visible?
>
> Yes. You can see it in the “window” menu.
>
> > Does the user recognize the action, even if the label is on screen?
>
> No. All test participants fail to discover this action. They expect
> this action to be in “preferences.”
>
> > Will the user understand the feedback?
>
> Yes, I believe participants will know the action works as they expect,
> but they prefer to have “single-window mode” activated by default.
>
> > Analysis – why does the breakdown happen?
>
> Quoting my textbook [1], default values:
> - are the initial state of the program.
> - are the best values, reflecting the most common use cases you
> observe. (Users don’t need to change them.)
> Many users don’t change them.
>
> Being completely new to the software, participants did not change the
> default values. Default values did not reflect their expectation.
>
> > Propose a solution!
>
> GIMP should be in single-window mode “by default.”

This is now the case in GIMP 2.10.

> > Quote your participants!
>
> - “Don’t have two windows. Use one window for the whole program.”
> - “When you open the software, you want to focus on the software. You
> don’t want to see gaps everywhere.”
>
> ================================
> Issue 2: accepting invalid input
> ================================

Ok so here, this is where I am starting to not understand much. As an 
advice, I think it is important to have the same terminology (either use 
the terminology seen in GIMP interface or in our documentions, or make a 
dictionnary with screenshots so that we understand what is your 
terminology at least) so that we understand what you say:

> When finding the pen tool, participants D and E first looked at the
> brush and texture panel, though it is not required for the task. In

I am assuming what you call "panel" is our dockables (cf. docs: 
https://docs.gimp.org/2.8/en/gimp-concepts-docks.html as well as menu 
labels). Otherwise I don't understand what you are talking about.

Also for the pen tool, brush or texture dockables can be useful, though 
I don't understand why the pen tool could be useful for the given task 
(transform a part of the image into B&W). But well, why not. :-)

> GIMP, it is important to define a selection before using the brush and
> texture panel.

I don't understand this. Why do you have to make a selection for the 
brush/texture dockables (still assuming panel == dockable)?

> The panel is active when a path is in the process of
> being drawn.

A path? I am assuming you are not using the vector tool (which is where 
we use the work "path"). You are probably refering to free selection 
(Free Select tool)?

> A participant interacted with the panel without a
> well-defined region, but this action did not change anything in the
> image, or change the property of the tool.
>
> > Does the user think of the right thing at the right time?
>
> No. The panel lacks a clear title. The participant thinks the panel is
> some kind of filter.

Hmmm… it is possible to display the dockable titles. This may not be 
done by default probably because of the place it would take, yet this is 
definitely an option available in GIMP.

> > Does the user understand the feedback?
>
> No. The panel shows an option is selected, but the option does not
> make sense in the current mode. The user expects a grayed-out panel.
>
> > Analysis – why does the breakdown happen?
>
> No signifier: the panel (or tab) lacks a title.
> No constraint: GIMP accepts input that does not make sense in the 
> current mode.
>
> > Propose a solution!
>
> Give each tab a visible title, in addition to the hover text. You can
> say: brush, texture, gradient.
> Gray out options that do not make sense in the current mode. When a
> path is still being drawn, gray out the brush, texture and gradient
> panels.

Ok let's go with my assumptions that you are talking about dockables and 
that "path" means a free selection in progress.

The brush, texture and gradient dockables are usable, whether or not a 
selection is present. This is indeed not changing something in the 
image. This is simply changing your selected brush, texture or gradient 
respectively (which will matter when you will use a tool needing these 
features, for instance the brush tool, etc.). Being in the process of 
making a selection simply has no relationship with these dockables, but 
that doesn't mean they are useless and therefore they should not be 
deactivated/grayed out.

> ==================================
> Issue 3: deviation from convention
> ==================================
>
> * By convention, I mean the previous knowledge which the user has
> acquired by using other similar software.
>
> Participant D could not close the path to form an area. She clicked on
> the first vertex, but the path did not close. She then dragged the
> vertices, hoping to make them “stick” together, but the path did not
> close. Getting stuck, she was given a hint to right-click. Following
> the hint, she converted the path to a closed region.

Hmmm… I don't get this one. Whether you are indeed talking about the 
Paths tool or in fact about the Free Select tool, you don't right click 
to close the shape. You must indeed just click the first point (which is 
what you said D was trying to do but it was not working? We'd need to 
know more to understand why).
Are you talking about something else?

> > Does the user think of the right thing at the right time?
>
> Yes. She wants to turn a closed loop into a region.
>
> > Is the action visible?
>
> Yes. The action to make a closed region is in the “tool options.”
>
> > Does the user recognize the action, even if the label is on screen?
>
> No. The participant tried to close the region by double-clicking and
> dragging the vertices, later got stuck and asked for a hint.
>
> > Does the user understand the feedback?
>
> Yes.
>
> > Analysis – why does the breakdown happen?
>
> The participant uses what she learned somewhere else. The action of
> clicking on the first vertex, or bringing two vertices closer, can be
> used to close a path in other programs, such as Inkscape and
> Photoshop.

Well as said, on GIMP too. But maybe I just don't understand what tool 
you are actually talking about.

> > Propose a solution!
>
> - Show a “four-side arrow” when the cursor hovers on a vertex.
> - Leverage what the user has previously learned! Implement a
> double-click handler that closes the loop, since dragging on an
> existing vertex moves it.
> - Make two vertices “stick” together when they are brought close.
>
> =================================
> Issue 4: lacking error prevention
> =================================
>
> Participant A makes high-quality selection, but GIMP removes his
> region when he clicks on tool to see what it says.

There are no cases when a selection disappears when you change tools. I 
could see cases when you could think that, but it would be in GIMP 2.10 
(a common issue with Free Select tool, which is now fixed) and you say 
you were using GIMP 2.8.  So I am a bit lost.

Basically selection are parts of the image (an existing selection will 
even be saved in the XCF), even though they don't change the render, and 
they are not tied to the fact that a selection tool is in usage.

> > Propose a solution!
>
> - When appropriate, show a confirmation dialog before deleting the
> user’s selection.

As Gary Curzi said, this is a bad idea. Selection are massively used in 
GIMP, they are created and removed many times in the life of an artwork. 
Therefore if each time you were removing a selection, you had a popup, 
it would be a very bad experience.

Moreover, still as said by Gary Curzi, undo works well on getting back a 
deleted selection (as I said earlier: the selection is part of the 
image; therefore undo/redo works on them).

> - When another tool kicks in, delete the selection only after the user
> clicks on the canvas with the tool.

No selections are ever deleted when switching tools.

> > Quote your participants!
>
> (Pre-interview) “When I have 300 photos, I don’t have 300 hours to
> edit them. I apply filters and move on. Sometimes I use pre-recorded
> settings.”

We have settings save/load in GIMP filters. We don't have macros (yet) 
though.

> ==================================================
> Issue 5: non-descriptive names and lack of presets
> ==================================================
>
> Participant C cannot tell the difference between “invert” and “value
> invert,” until he reads the tooltip text. He cannot understand what
> “colorize” and “colorify” mean until he opens the dialog.
>
> “Colorize” controls the hue, saturation and brightness. He could have
> achieve the black-and-white effect by turning down the saturation, but
> he adjusted the hue first, and he was not satisfied with the result.
> He expanded the dropdown for presets, but there wasn’t any preset.
>
> > Propose a solution!
>
> Rephrase the options. For value invert, how about “brightness invert,”
> as said in the tooltip description.
> Value invert ==> Brightness invert
> Colorize     ==> Hue, Saturation, Brightness
> Colorify     ==> Colored-glass

Some things could definitely be better named. I think we should also try 
to find the right compromise between some well known color science 
naming and more "common human" descriptive names.

Though in your case, C is described as an occasional user of such 
programs. So we should also be careful because it is normal that 
occasional users don't know names of very common filters or algorithms 
and we are not going to change color science for them.
In the given examples for instance, "value" is a common name in color 
science (see https://en.wikipedia.org/wiki/Lightness) and has no random 
meaning. The fact that C doesn't know what it means should not mean we 
should redefine what some people have worked on for years. Now here 
"Value" and "Brightness" are related but I don't think they are synonyms 
as value is just one kind of brightness representation, if not mistaken 
(others more knowledgeable on color science, feel free to correct).

What also comes into account sometimes might be how a given very common 
filter is called on some other similar programs, as well as historical 
names in GIMP (people don't like when we change names, so it has to be 
done sparingly).

Still I agree that many things are badly named in GIMP. I would suggest 
to make bug reports if you have suggestions for better naming of various 
things (just do one report for one item name change, not a report 
"change a bunch of stuff"; what we call "actionnable" changes). It would 
need to be well documented and argumented though (not "C who knows 
nothing about color science did not understand a well-defined term").

> Don’t leave out the presets. You can start with making simple presets,
> like “desaturate”.

I think that presets are mostly for user-made filters, but why not have 
filters of common usage… We should be careful though not to overdo it. 
Not everyone has the same needs.

Moreover it looks like here C was making a beginner mistake by trying to 
play with Hue to get greyscale values. I think that anyone even just a 
little knowledgeable on pixel manipulation knows that you should play 
with saturation for this and really don't need a preset for this (this 
would be even slower).

Anyway thanks for this report. It is interesting, though in the end I'm 
not so sure what to conclude of it. Also the more it goes, the less I 
believe that GIMP is a good candidate for user testing of the form "just 
drop someone who knows nothing about image manipulation and expect this 
person to do magic". GIMP is a complicated software, that's also because 
it does very powerful things. I don't expect a complete beginner who 
never read any documentation to be an expert in 1 hour. It just makes no 
sense. This is the same for any software of the same category by the 
way. So for instance in your user testing, I'd be more interested into 
what A and B (the only 2 experienced people) would have to say.

Maybe a more interesting user testing with beginners should be done like 
this: they first follow a short introduction/lesson to common and 
limited set of features with examples of usage. Then they are asked to 
do a given task with what they have learned and the tools they were just 
explained. This would be much more interesting maybe.

Jehan

> ==========
> References
> ==========
>
> [1] Ko, A. (n.d.). “How to design user interfaces”
> <https://faculty.washington.edu/ajko/books/design-methods/how-to-design-user-interfaces.html
> <https://faculty.washington.edu/ajko/books/design-methods/how-to-design-user-interfaces.html>>
>
>
> Best regards,
>
>   Flynn Liu
> _______________________________________________
> gimp-gui-list mailing list
> gimp-gui-list gnome org
> https://mail.gnome.org/mailman/listinfo/gimp-gui-list

--
ZeMarmot open animation film
http://film.zemarmot.net
Liberapay: https://liberapay.com/ZeMarmot/
Patreon: https://patreon.com/zemarmot
Tipeee: https://www.tipeee.com/zemarmot