GNOME.org
 

Re: [Usability] Usability meeting minutes


Long rambling comments below :)

On 27 Oct 2008, at 18:56, Stormy Peters wrote:
In the first few cycles of GNOME 2, there were UI reviews. They looked at new dialogs, gave feedback, kept a good state in GNOME. But no longer have 
people that have time to do that.
There were another couple of reasons we stopped doing UI reviews-- one of which was that we never managed to finish one. At best, we usually managed to review some parts of three or four applications before time ran out, and/or the depressing reality of trying to do UI reviews over IRC hit us and we all went home :)
The main reason, though, was that they were beginning to perpetuate the misconception that usability is something you just do at the end of a cycle to add some lipstick to the pig. We vowed to engage developers at the beginning of the development cycle instead, and work with them (and users) to help validate their plans and assist them with designs. That was about the same time that the number of active usability practitioners in the community started dropping off, though, so it never really happened.
One alternative that we haven't tried yet might be to have 'usability days' throughout the dev cycle, much as some of the larger desktop apps have their own bug days. A day where hackers and usability folks meet on #usability, and pick off bugzilla bugs that have been tagged with the usability keyword. They needn't be fixed there and then (I'd expect most of the ones that could be fixed on the spot to be of the less-useful pig and lipstick variety again, although of course they do need fixing too). Just so long as somebody agrees to work on them, and a usability person agrees to help them out.
3 years ago Novell sponsored the betterdesktop.org - a usability study. The goal was to get some feedback on what is good from outside the project, people without prior experience with GNOME. The community was not involved and the conclusions were not discussed in the community, so it didn't have 
the impact that it could have had.
Yes, that was a pity. One thing they hopefully did prove to the community is that you don't need a lab to do usability testing... they put together their own mobile lab and went all round the world with it IIRC. We certainly need to find more ways to do that sort of budget usability testing, because with most companies tightening their belts at the moment, subsidising GNOME usability studies in their labs probably isn't going to be high on their agenda.
While we have a lot of developers that are aware of usability, developers are not designers, they make mistakes and need help. For example, on the usability mailing list we don't have many people helping anymore. Yet people 
still come for feedback. Missing designers.
Yep. And while it's always difficult to attract people with non- coding skills to an open source project at the best of times, usability folks tend to be particularly hard...
While a lot of hackers enjoy writing code in their spare time, and many artists enjoy doing likewise with cool backgrounds or icons, very few people 'do usability' as an entertaining diversion of an evening. Nor, unlike those other two disciplines, do many people teach themselves how to do it for fun in the first place.
So immediately, we're looking at a potential pool of people who are most likely employed to work on the usability of something other than GNOME during the day, and who don't have much motivation to work on the usability of anything else in their spare time.
For those who do make the leap, as they're not usually contributing code or patches, there aren't so many things they can jump in and do straight away, nor is it as easy to prove that they're good at what they do. They can file usability issues in bugzilla, but unless they're a recognised 'name', they're somewhat less likely to be taken seriously without some backup from the other usability folks.
Now and again people do introduce themselves on the list as studying usability or some such, and many just disappear again without trace because we didn't usefully engage them. We need to stop that happening.
We no longer do usability reviews during development cycle. Last one was 2.10 or 2.12 and we have lost some consistency and the new modules could 
have used some review. Lacking designers!
(2.14, to be precise...) As noted earlier, I don't think the lack of designers is really the issue here. UI reviews were carried out post feature-freeze, so there was no real 'design' happening in those anyway.
Don't have a project wide vision of what the GNOME desktop should look like. Don't have people leading and helping understand why this is good or bad and help make decisions. For example, GNOME menu was proposed to be 
included in GNOME. Big change in how applications are launched. Some
discussion, but no concensus because we don't have enough people with
usability and designer backgrounds to say what is good.
"People with usability and designer backgrounds" can't really tell anyone what's "good" either, at least not without feedback from users. At best, they can use their prior experience to make informed guesses, but guesses is all they are at such an early stage.
We've threatened for many years to develop some personas[1] for use when developing new features. Perhaps now is the time to seriously address that. However, it's generally accepted that personas are only useful tools if they're based on actual user research and interviews, so sitting around and making them up ourselves wouldn't cut it.
That, I think, is a key point for all our usability-focused work: if we're to progress, we need to get out of the habit of designing things purely as the result of discussions among ourselves. You can't design great products without involving users.
In particular, we do have some insight into the type of people who use GNOME by choice-- we all fall into that category, albeit the (small?) subset who have the time/inclination/ability to actively engage with its developers and try to improve it. But there's also a growing audience of people out there who *have* to use GNOME whether they like it or not, on school or work computers, or in internet cafes, public libraries etc. How much do we really know about them?
GTK+ is older than GNOME and does not have the same HIG. Standard GTK+ dialog had to be fixed in every application. Changing now as GTK+ folks are 
involved in GNOME but has been an issue in the past.
Glade could probably have been rather more helpful in the meantime, too, by shielding designers from any non-optimal gtk+ defaults. I don't know how much time I've wasted over the years left- or right- aligning text labels that always default to centre alignment, or adding the same HIG-compliant row and column spacing to tables and boxes that Glade always adds to my dialogs with a spacing of zero.
Community is well educated about usability but when someone new joins, there's no good and easy documentation or tutorial that explains why we do things this way or to teach them what's good and what's bad from a usability 
perspective.
Not sure I necessarily agree with that. The HIG itself has a pretty good, simple introduction to general usability[2], and the rest of the HIG, while certainly not a light bedtime read, accompanies guidelines with rationale where possible. That's a pretty good resource for newcomers, and any general usability questions to the mailing list are usually well-answered too, in my experience-- often more so than people looking for specific design advice, because that's always harder :)
Try to change as few central things as possible. Set goals for next few years. For example, the panel might need some updates. File manager - like Federico's talk about finding documents. Do we want a timeline view? Find on 
text instead of folders? Have some broad discussions.
And ask users. And preferably observe them too, because what users say and what they do are often different :)
Usability studies. Having some sponsored would help. Dave Richards had 
feedback from users on what's wrong and what works.
I'd also really like to see more of the simple usability work done by developers.
Of course it would require some hand-holding/tutorials/whatever from experienced usability folks to start with, but really, it shouldn't be that hard to get everyone involved, testing early and often. Use paper prototypes, working prototypes, test with friends, relatives, students, user groups. Video the results if possible (but not essential). Collate the results on a suitable website (would need consent if participants were identifiable in videos.)
If somebody could write a *nix version of something like Silverback[4], that would be a great tool to have in our armoury, too. (Slight mixed metaphor there...)
Need to make sure we have a solid foundation. Current HIG is in draft form, haven't been formaly approved. With things like Clutter, OpenGL, there's a lot of usability change and a chance to do new interfaces, and we should 
invest how those would be in the HIG.
Not sure what's meant by the 'draft' comment here-- we've always had a version of the HIG that's considered stable (currently <http://library.gnome.org/devel/hig-book/stable/index.html.en >), and a work-in-progress version that will go on to form the basis of the next stable version (currently <http://developer.gnome.org/projects/gup/hig/draft_hig_new/ >).
There is, of course, little doubt that the HIG needs updating. The first step in that process is probably to decide what developers want the HIG to be:
* A book, like it is now? (If so, should we start selling hard copies via Lulu or something? I'd buy one...)
* Reduced right down to a collection of UI patterns, with accompanying code samples? 

* An online help document with hooks into Glade?

* Something else?
There is probably a need for a mobile device HIG, too, but that's likely to be a separate document (just as Apple, Microsoft and Sun have found to be necessary with theirs). The requirements for touch/ tablet interfaces are just too different from desktop interfaces.
Whatever form of guidelines we provide, once again it would be good if they were more informed by user feedback. A good example of this is the current discussion on this list about tab design; we have many applications implementing different tab functionality, so what should the HIG be recommending? How do we decide what functions should be set in stone by the tab widget, and which are subjective enough to require tweakability via desktop- or application-level preferences? We're deluding ourselves if we think we can second-guess our users without asking them.
Scratchpad for thoughts on the next version of the HIG: <http://live.gnome.org/UsabilityProject/HIG >.
Are we using the wrong tools to attract designers? Not on IRC, mailing 
lists, ...
We probably need to be reaching out more actively to them. One possibility might be to become more involved in the OpenUsability project <http://openusability.org> community, which has so far assisted with more KDE apps than GNOME apps, AFAICT. 


Set some sort of milestone inside of release cycle where we call for
designers to come try things and get advice.
Designers really shouldn't be "coming to try things", except perhaps some very early proof-of-concept designs. They should be driving the design of those "things" in the first place, and working with developers throughout the development cycle to refine those designs.
(Sun) Been bad about getting feedback from desktop users that aren't part of our team. How do we encourage getting feedback from university users or 
people at work.
We certainly miss a lot of casual feedback from various sources-- for example, from people who might try GNOME once via a magazine coverdisk or conference giveaway, and never try it again. How do we find out what they didn't like?
We need to make it easier for casual and non-technical users to submit their feedback. One idea that's been used elsewhere before now would be to add something like a 'feedback' applet to the panel. Click the smiley face icon to send positive feedback about the running application, or the sad face icon to send negative feedback. ("Feedback" could be anything: something that was easy/hard to do, something that's got quicker/slower since the last time you tried it, some peripheral working/not working). The applet could take a screenshot of the running app (or desktop) as well, if relevant, that could be included in the report with the user's consent. (Of course, we'd have to figure out where to send the feedback and what to do with it... just brainstorming here!)
Another possibility would be to instrument selected applications during development cycles, and gather raw usage information that way, as the inGIMP folks have done with GIMP[3]. (The classic problem with that sort of thing is knowing what to do with all the raw data, though...)
It's also just not possible for all of us to monitor every source of good information, which is really quite thinly spread around our community. Good ideas might scroll past unnoticed on IRC while we're in bed, and how many of us have time to actively read the GNOME Footnotes forums as long as there's no useful RSS feed--the one place where many 'real users' actually post? Not many, judging by the number of posts that go unanswered. And good luck keeping track of everything that comes and goes on live.gnome.org...
Perhaps we need a Feedback Squad, akin to a Bug Squad, who monitor all those sources (as well as liasing with user groups, and users of GNOME installations in schools/universities/offices etc.) and get it to the right places/people?
If you have money to spend, send designer or user interface engineer to large deployments of GNOME - user testing in field. City of Largo - GNOME - Dave Richards - have problems. Would have been useful to get it earlier. How 
would we generate the list of people?
If you look at how large companies do usability testing, they typically have a database of people who've participated in usability studies before, and people who've expressed an interest in taking part, along with some basic demographic information about them. When a specific usability study comes up, a recruiter will use further screening criteria specific to that study, to recruit participants from the database for that study. And when they take part, they'll be given a small gift or financial incentive in return for their time.
Perhaps we need to build up such a database? We could recruit testers from our user base via appeals on www.gnome.org, Linux magazine ads/ articles, mailshots to companies with large GNOME installations, a link in the "About GNOME" dialog etc. Store respondents' details on file, and select via screener questionnaires when we want to test a new feature. Offer GNOME swag to test participants.
The 'studies' themselves needn't take place in a fancy lab, there are plenty of other ways to run a usability study or focus group. Face-to- face is certainly preferable where possible, but remotely via VNC/ videoconf, online questionnaires, online card sorting, specially- instrumented versions of software etc. are all possibilities too, depending on what needs to be tested. 

Cheeri,
Calum.
[1] <http://www.cooper.com/journal/2001/08/ perfecting_your_personas.html> 
[2] <http://library.gnome.org/devel/hig-book/stable/principles.html.en>
[3] <http://ingimp.org/>
[4] <http://silverbackapp.com/>

--
CALUM BENSON, Usability Engineer       Sun Microsystems Ireland
mailto:calum benson sun com            GNOME Desktop Team
http://blogs.sun.com/calum             +353 1 819 9771

Any opinions are personal and not necessarily those of Sun Microsystems
[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]