All projects: Gel, Jobs, Good Todo, Games, Uncle Mark, Blog, Bit Literacy
Four Words to Improve User Research
Like any good user experience practitioner, I often spend time face-to-face with users of clients' products, services, and websites.
Unlike many practitioners, I use a research method that is a bit different from the established teaching of usability and HCI.
I'll admit upfront that I'm biased towards this method (note: always be wary of gurus whose ego is bigger than the building you're sitting in). But I'm biased because I've seen the results this method creates, both in the data that's generated and in the *business results* the client enjoys afterwards.
The method is the "listening lab": a more open-ended version of the traditional usability test. Listening labs generate *strategic* findings - not just tactics - and point the way to measurable *business* results - not just usability results like task success or time-on-task.
In their setup, listening labs are familiar: one-on-one sessions with a moderator and user, in front of a PC, typically in a lab facility with a one-way mirror, with observers sitting in the other room.
Listening labs differ from usability tests in one key way. This is the one simple thing that, if done properly, can transform the research and its results. Just four words.
FOUR WORDS TO IMPROVE USER RESEARCH:
Don't
define
tasks
beforehand.
When you walk into the testing facility as the moderator, or even as the client, *don't* have a script or list of tasks written out already. Maybe you were told to do so in a usability book, or by a "guru," or learned it at school. But don't do it.
Instead, when the test starts, interview the customer on how he or she relates to the website or product in question. (When, and why, do they typically use this service? What about competitors? What's the process they go through? What's a specific example from the recent past, or near future, that they will undertake with this service?) So, first understand their context: show that you're *listening*.
Then, with the customer's context in mind, give them a task to conduct - the same task they just described to you in the specific example. As they turn to the PC, ask them to keep talking, describe what they're thinking, and say why they're taking each action.
At this point, the customer hands you test data you never could get in a rigid, traditional usability test: tactical learnings, strategic thinking, and everything in between; almost none of which you could have *guessed* beforehand, had you written a script.
Because that's what a task list is, after all - the test moderator presuming to guess how customers use the service, without even talking to them first. It's the moderator setting a rigid framework, saying, "I know how and why customers are here."
But most websites are a strategic representation of the *business*. How can you presume to know how customers relate to the business, unless you ask them first? If your test moderator has a healthy case of ESP, or can bend spoons without touching them, by all means, define the tasks beforehand. Otherwise, try the listening lab method.
And remember that you *will* get tactical data. Listening labs *are* task-based - but only on the tasks most relevant to each user.
Strangely enough, most customers (within each customer segment) will VOLUNTARILY come up with the same tasks, and the same feedback, on the site - without any leading from the moderator.
I just finished two days of listening labs for a global Fortune 500 corporation. Creative Good is creating the customer experience strategy for the entire site, so in the listening labs we had respondents from all their major customer segments. By the fourth respondent in each segment, we were seeing the same tasks, the same feedback, the same results - created voluntarily by the customers themselves.
Here are some next steps:
- For you: In your next round of customer research, don't define tasks beforehand. Let the customers lead. (Another good four-word phrase, come to think of it.)
- For me: In my copious spare time, I'll try to write more about listening labs, since this is the topic I get asked about most (along with measuring business results) at the seminars, conferences, and companies I speak at.
Why the hostility toward a time-tested and reliable tool in the toolbox? Usability testing and listening labs BOTH have their merits, so I don't appreciate the unnecessarily judgmental language in this post. Seems to me that listening labs are great if you are still in the discovery phase, but if you want to provide tangible proof that the product you built supports the tasks that were previously identified through user research, usability testing with pre-defined tasks is the way to go.
Good point Zora. One other add: Sometimes its impossible to bring in customers and have an open ended discussion/use of your product with them (for example, someone who uses your highly-specialized pro tool only during a particular time of the year). In that case, we travel to the customer's office or home office (place of use) and just observe them doing what they do. So the "Listening Lab" need not be in a lab or even at your place of work. It could be anywhere, including the customer's natural place of use (many times the best place).
I also wanted to ask the question on why the name "Listening" Labs? It goes against the common learning technique we all know: that you should not LISTEN to what your customers SAY, but rather WATCH what they DO. These are often 2 very different things. Your definition of "Listening Labs" process does involve watching and doing, so I think the problem is in the title only.
Is it time to change "Listening Lab" title to "Contextual Walkthrough" or "Exploration Lab." If you are really into the ring of the name, maybe "Contextual Expedition" or "Exploration Expose'." :)
Zora - What Mark is describing in the post is strictly User Research which should initially be carried out in the discovery phase of a project. Whereas you seem to be talking about User Testing in which there does need to be a lot more structure in setting senarios etc for the user to complete.
Mark does have a very good point that User Research can sometimes be too structured and result in the facilitator/observer leading the user through the session rather than visa versa. Which can ultimatly lead to tainted reseach.
Phil - I agree that the name Listening labs isn't ideal as we are also watching and taking notes. I quite like "Exploration Labs" or "Discovery Labs" if we're going to have a vote on it.