|
-- Tim McConnell <tmick1 earthlink net> |
Send Usability mailing list submissions to usability gnome org To subscribe or unsubscribe via the World Wide Web, visit http://mail.gnome.org/mailman/listinfo/usability or, via email, send a message with subject or body 'help' to usability-request gnome org You can reach the person managing the list at usability-owner gnome org When replying, please edit your Subject line so it is more specific than "Re: Contents of Usability digest..." Today's Topics: 1. Re: Usability Digest, Vol 55, Issue 5 (Allan Day) 2. Re: Prototyping the next generation panel (Jacob Beauregard) ---------------------------------------------------------------------- Message: 1 Date: Sat, 08 Nov 2008 12:20:33 +0000 From: Allan Day <allanpday gmail com> Subject: Re: [Usability] Usability Digest, Vol 55, Issue 5 To: Jacob Beauregard <deadowlsurvivor gmail com> Cc: "usability gnome org" <usability gnome org> Message-ID: <1226146833 6132 13 camel blackbox> Content-Type: text/plain Hi Jacob, I get the feeling that we're not going to be able to agree on this! ;) I think we're both coming from very different research traditions - it's no simple matter bridging those kinds of divides. Just to explain - I'm very much in favour of in-depth qualitative research - through semi-structured interviews, observations, ethnography, etc. I actually think those kinds of approaches could be very beneficial for GNOME, but that's another matter... Why don't we try and start at the beginning? We should be starting with more general questions: What should be the purpose of our research be? What kind of resources do we have available with which to do that research? Having said that, I am still concerned about the possible consequences of what you're proposing (more below)... > >>>> The biggest problem that's going to occur is that most people have an > >>>> extremely strong preconceived notion of how a desktop should behave, > >>>> and while you want to have representation this group, it can't be the > >>>> only group that's represented if you really want to improve the > >>>> desktop experience. > >>>> > >>> Very true... probably makes it more important for us to consider the > >>> learnability of any proposed designs, too. We can't necessarily > >>> expect people to "get it" the first time they use a completely new > >>> desktop, but if they're comfortable and productive within a week, then > >>> we might be onto something. If it takes them six months, we're > >>> probably not. > >>> > > > > It'll be important to get background information from participants about > > their previous desktop experience. Conducting observations, we should > > also be looking out to see at which points people's behaviour is > > directed by habits picked up from existing systems. > > > It would be a mistake to deny that. > > > >>>> Fortunately, I recently discovered a clever solution to this. You > >>>> could weed a lot of these people out by building a survey with a very > >>>> open-ended question that suggests an answer. Then you can see who > >>>> responds with the suggested answer. > >>>> > >>>> Example survey question where the results were usually echoed from > >>>> the suggestion, however with exceptions: > >>>> -------------- > >>>> What interests you in the field of Computer Science? Why? > >>>> (Ex. Do you enjoy creating things? Do you enjoy knowing how stuff > >>>> works?) > >>>> -------------- > >>>> Then you can find the people that don't simply echo, and at the same > >>>> time you can also represent the people that do echo, it's just easier > >>>> to find the other people that don't echo way. > >>>> > >>> Sounds like a good idea-- the screener questionnaire is certainly an > >>> important part of selecting participants for any study. But I've > >>> always been lucky enough to have other people around to do that part, > >>> so I'm not really all that qualified to comment :) > >>> > > > > Personally, I'm unsure about how useful a pre-questionnaire would be in > > this particular respect, since I wouldn't expect there to be a simple > > relationship between people's conscious understandings and how they use > > the desktop. Let's sit them down at the prototype with a task (or > > whatever it is that we end up using) and see what happens. If we need > > to, we can ask questions about their actions either as they go or > > afterwards. > > > I wouldn't consider it very expensive to find out if you're unsure, and > I would bet my life that there is a very strong connection. In fact, > with a pre-questionnaire, you can save a lot of time and money (if > you're spending money to do this). On the connection between self-understandings and behaviour - I think we'll have to agree to disagree! :) (Though I would say that my take on this relationship has been well documented in the past - indeed, the proposition that there isn't a simple link between the two forms the basis of many major academic research traditions.) On a more practice note: if, as you say, the majority of people will not react well to a redesigned desktop environment, then we shouldn't ignore them - we should pay these people special attention in order to understand how we can make sure the redesign is as well suited to them as possible. > > I'd have concerns about classifying people according to a predefined > > schema (if that is what you're suggesting). In this situation, it would > > be much better to generate our own groupings through observation and > > analysis, rather than relying on pre-existing conceptualisations of > > behaviour. > > > People are already classified according to predefined schema. Yes - but if we can produce our own analyses, rather than relying on existing ones, then that might help with producing a desktop experience that is truly original. Plus, I don't like the idea of reproducing other peoples' classifications. ;) > Go ask a > marketer for a software or hardware company if you have any doubts. You > think Dell would sell a gaming PC if they didn't segment the market to > include "gamers?" Do you think they would sell laptops if they didn't > segment the market to include "mobile users?" > > Some categories: > --Power users > --Casual users > --Mobile users > --Desktop users > --Home users > --Enterprise users > --Hardcore gamers > --Casual gamers > --Studio users > --Users that are disabled (hard-of-sight/hard-of-hearing/motor-impaired) > > Yes, there are a million ways to categorize users. You want to segment > users based on how they use their desktop. Before you do that, you want > to segment users based on what they'd like to be able to do with their > desktop. Some of these groups are a lot bigger than others. A > pre-questionnaire allows you to make sure all of, or most of, the market > segments are represented with minimal cost (i.e. you won't end up > interviewing fourteen power users and a single casual user if 15 is the > size of your sample). The best resource to aid you in segmenting the > user population, and designing a pre-questionnaire, once again, would be > someone in a marketing department. I just don't think that marketing research is helpful in relation to the redesign effort - its purpose is quite different from what we need. > If you're asking a question during an interview, "how do you improve the > desktop experience?" most responses will be bound to the user's past > experiences. There's a segment of users that aren't nearly as bound to > their past experiences and they would likely be a more reliable resource > for brainstorming, whereas the other group would likely be a more > reliable source to measure practicality. I wouldn't expect research participants to have the kind of specialist knowledge that would allow them to answer interview questions like that. Instead, I would imagine that exploring new possibilities would come out of a dialogue between researcher and participant - going through exercises with a research participant, you could talk about particular difficulties with participants and suggest possible solutions, for example. This is one place where involving developers would be great - researchers could act as translators between developers and users within an iterative design process (open source approach to research, anyone?!). <snip> > What I'm trying to get at is > that the question isn't necessarily how do you use your desktop, but in > what ways can you use your desktop outside of the boundaries of what's > already defined that would make it more useful to you. Then you focus on > HOW can an idea be made both practical and as usable as possible. </snip> I completely agree - we need to ensure that the research we do doesn't uncritically reproduce existing design patterns. We also need to be able to explore new design possibilities. I think we can do this by using observations of desktop usage (or prototype usage) as a starting point from which to talk with users about what they do with their machines. That ordering (practice, then speech) is an important one, IMO, since it grounds the discussion. You can say - 'why did you do that?', or 'why didn't you do that?'. Also, it is worth remembering that analysis would in no way be restricted to the description of the behaviour that was observed. Best, Allan ------------------------------ Message: 2 Date: Sun, 09 Nov 2008 02:04:44 -0500 From: Jacob Beauregard <deadowlsurvivor gmail com> Subject: Re: [Usability] Prototyping the next generation panel To: allanpday gmail com Cc: "usability gnome org" <usability gnome org> Message-ID: <49168B8C 30607 gmail com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed I'm trying to switch the thread title back, please reciprocate. Allan Day wrote: > Hi Jacob, > > I get the feeling that we're not going to be able to agree on this! ;) > I think we're both coming from very different research traditions - it's > no simple matter bridging those kinds of divides. Just to explain - I'm > very much in favour of in-depth qualitative research - through > semi-structured interviews, observations, ethnography, etc. I actually > think those kinds of approaches could be very beneficial for GNOME, but > that's another matter... > I don't consider there to be sides in research. The entire goal is to get as much relevant information at the least cost possible. > Why don't we try and start at the beginning? We should be starting with > more general questions: What should be the purpose of our research be? > What kind of resources do we have available with which to do that > research? > > Having said that, I am still concerned about the possible consequences > of what you're proposing (more below)... > > You're still not understanding me, as you're disagreeing with something else entirely. >>>>>> The biggest problem that's going to occur is that most people have an >>>>>> extremely strong preconceived notion of how a desktop should behave, >>>>>> and while you want to have representation this group, it can't be the >>>>>> only group that's represented if you really want to improve the >>>>>> desktop experience. >>>>>> >>>>>> >>>>> Very true... probably makes it more important for us to consider the >>>>> learnability of any proposed designs, too. We can't necessarily >>>>> expect people to "get it" the first time they use a completely new >>>>> desktop, but if they're comfortable and productive within a week, then >>>>> we might be onto something. If it takes them six months, we're >>>>> probably not. >>>>> >>>>> >>> It'll be important to get background information from participants about >>> their previous desktop experience. Conducting observations, we should >>> also be looking out to see at which points people's behaviour is >>> directed by habits picked up from existing systems. >>> >>> >> It would be a mistake to deny that. >> >>> >>> >>>>>> Fortunately, I recently discovered a clever solution to this. You >>>>>> could weed a lot of these people out by building a survey with a very >>>>>> open-ended question that suggests an answer. Then you can see who >>>>>> responds with the suggested answer. >>>>>> >>>>>> Example survey question where the results were usually echoed from >>>>>> the suggestion, however with exceptions: >>>>>> -------------- >>>>>> What interests you in the field of Computer Science? Why? >>>>>> (Ex. Do you enjoy creating things? Do you enjoy knowing how stuff >>>>>> works?) >>>>>> -------------- >>>>>> Then you can find the people that don't simply echo, and at the same >>>>>> time you can also represent the people that do echo, it's just easier >>>>>> to find the other people that don't echo way. >>>>>> >>>>>> >>>>> Sounds like a good idea-- the screener questionnaire is certainly an >>>>> important part of selecting participants for any study. But I've >>>>> always been lucky enough to have other people around to do that part, >>>>> so I'm not really all that qualified to comment :) >>>>> >>>>> >>> Personally, I'm unsure about how useful a pre-questionnaire would be in >>> this particular respect, since I wouldn't expect there to be a simple >>> relationship between people's conscious understandings and how they use >>> the desktop. Let's sit them down at the prototype with a task (or >>> whatever it is that we end up using) and see what happens. If we need >>> to, we can ask questions about their actions either as they go or >>> afterwards. >>> >>> >> I wouldn't consider it very expensive to find out if you're unsure, and >> I would bet my life that there is a very strong connection. In fact, >> with a pre-questionnaire, you can save a lot of time and money (if >> you're spending money to do this). >> > > On the connection between self-understandings and behaviour - I think > we'll have to agree to disagree! :) (Though I would say that my take on > this relationship has been well documented in the past - indeed, the > proposition that there isn't a simple link between the two forms the > basis of many major academic research traditions.) > > On a more practice note: if, as you say, the majority of people will not > react well to a redesigned desktop environment, then we shouldn't ignore > them - we should pay these people special attention in order to > understand how we can make sure the redesign is as well suited to them > as possible. > > I'm not saying that a majority of people won't react well to a new desktop environment. I'm just saying they have a lot of people would have no idea on how they would change the desktop environment so that it would better suit them because they've been doing the same thing for so long. >>> I'd have concerns about classifying people according to a predefined >>> schema (if that is what you're suggesting). In this situation, it would >>> be much better to generate our own groupings through observation and >>> analysis, rather than relying on pre-existing conceptualisations of >>> behaviour. >>> >>> >> People are already classified according to predefined schema. >> > > Yes - but if we can produce our own analyses, rather than relying on > existing ones, then that might help with producing a desktop experience > that is truly original. Plus, I don't like the idea of reproducing other > peoples' classifications. ;) > > I'm not saying not to produce your own analyses, just not to ignore past analyses. Ignoring past analyses is also extremely costly to the entire development process. >> Go ask a >> marketer for a software or hardware company if you have any doubts. You >> think Dell would sell a gaming PC if they didn't segment the market to >> include "gamers?" Do you think they would sell laptops if they didn't >> segment the market to include "mobile users?" >> >> Some categories: >> --Power users >> --Casual users >> --Mobile users >> --Desktop users >> --Home users >> --Enterprise users >> --Hardcore gamers >> --Casual gamers >> --Studio users >> --Users that are disabled (hard-of-sight/hard-of-hearing/motor-impaired) >> >> Yes, there are a million ways to categorize users. You want to segment >> users based on how they use their desktop. Before you do that, you want >> to segment users based on what they'd like to be able to do with their >> desktop. Some of these groups are a lot bigger than others. A >> pre-questionnaire allows you to make sure all of, or most of, the market >> segments are represented with minimal cost (i.e. you won't end up >> interviewing fourteen power users and a single casual user if 15 is the >> size of your sample). The best resource to aid you in segmenting the >> user population, and designing a pre-questionnaire, once again, would be >> someone in a marketing department. >> > > I just don't think that marketing research is helpful in relation to the > redesign effort - its purpose is quite different from what we need. > A marketing department is far more attuned to research tools such as surveys than most others in a business. The way the questions are worded in a survey will drastically affect the response pattern. Marketers usually have the most experience in an organization at wording surveys, so I'm just saying if you're making a survey and you don't have much experience doing so, try asking someone in a marketing department. If you're looking to understand how to segment users for the purpose of your study, same thing. Marketing departments are very good at those sorts of things because it's part of their day-to-day activities. > >> If you're asking a question during an interview, "how do you improve the >> desktop experience?" most responses will be bound to the user's past >> experiences. There's a segment of users that aren't nearly as bound to >> their past experiences and they would likely be a more reliable resource >> for brainstorming, whereas the other group would likely be a more >> reliable source to measure practicality. >> > > I wouldn't expect research participants to have the kind of specialist > knowledge that would allow them to answer interview questions like that. > Instead, I would imagine that exploring new possibilities would come out > of a dialogue between researcher and participant - going through > exercises with a research participant, you could talk about particular > difficulties with participants and suggest possible solutions, for > example. This is one place where involving developers would be great - > researchers could act as translators between developers and users within > an iterative design process (open source approach to research, > anyone?!). > > "how do you improve the desktop experience?" Yea, I guess the wording is off on that. I should have worded it, "how would you." It doesn't take a specialist to brainstorm new ideas. I'm just proposing a way of being more productive with brainstorming. In any case, there's a problem with the method you described above. Suggesting something to a participant can very easily bias the participant to your own ideas, and it's usually accidental when that happens. It's like when the police or prosecutors pose leading questions. It's best for responses to be unfiltered, preferably unaffected by prior suggestions, and to have as many as possible. There's plenty that's already been suggested, but it doesn't seem complete by any means, and could potentially be a lesser solution. > <snip> > >> What I'm trying to get at is >> that the question isn't necessarily how do you use your desktop, but in >> what ways can you use your desktop outside of the boundaries of what's >> already defined that would make it more useful to you. Then you focus on >> HOW can an idea be made both practical and as usable as possible. >> > </snip> > > I completely agree - we need to ensure that the research we do doesn't > uncritically reproduce existing design patterns. We also need to be able > to explore new design possibilities. I think we can do this by using > observations of desktop usage (or prototype usage) as a starting point > from which to talk with users about what they do with their machines. > That ordering (practice, then speech) is an important one, IMO, since it > grounds the discussion. You can say - 'why did you do that?', or 'why > didn't you do that?'. Also, it is worth remembering that analysis would > in no way be restricted to the description of the behaviour that was > observed. > This is what should be done at the beginning of any project: 1. Define the problem(s) 2. Determine requirements Where are there some problems defined?: http://www.vuntz.net/journal/2008/10/22/494-desktop-shell-from-the-user-experience-hackfest-general-overview http://live.gnome.org/Boston2008/GUIHackfest/WindowManagementAndMore I didn't actually go to Boston, so I have no idea what the direction is and why. For that reason, I believe it should be discussed a bit more. The problems are not nearly documented well enough, nor is the solution. My response to parts of the solution as-is. ------------------ "There should be one centralized way to display all sorts of notifications, ranging from system updates about battery time left to updates about new e-mails to updates about your friends' activity on the web to rss feed updates." That sounds like a recipe for another source of organizational mayhem for users if not implemented very carefully. I suppose notifications will include system stats, like battery power, and it would be nice to see task progress in there instead of just being notified when a task is complete. However, when thinking about centralized notifications, there is unyielding potential for how it's used, a ridiculous amount of gray area. ------------------- "The sidebar should come preconfigured with useful widgets, like a recent files selector/Journal widget, task list widget, most used applications widget, and some fun ones, like Rhythmbox current song album art display." A single sidebar is not nearly enough for all of the above, especially with that mockup. After you understand the problems, you want requirements. What can be employed for requirements discovery: ---existing research ---new research ---observations ---questionnaires ---interviews ---prototypes ---joint requirements planning ---and so on You can't exactly limit yourself to one type of research. > Best, > > Allan > > > ------------------------------ _______________________________________________ Usability mailing list Usability gnome org http://mail.gnome.org/mailman/listinfo/usability End of Usability Digest, Vol 55, Issue 9 ****************************************