Four Words to Improve User Research
Wednesday, October 1, 2003
by Mark Hurst
Get Good Experience by e-mail: e-mail update@goodexperience.com
Like any good user experience practitioner, I often spend time
face-to-face with users of clients' products, services, and
websites.
Unlike many practitioners, I use a research method that is a bit
different from the established teaching of usability and HCI.
I'll admit upfront that I'm biased towards this method (note: always
be wary of gurus whose ego is bigger than the building you're
sitting in). But I'm biased because I've seen the results this
method creates, both in the data that's generated and in the
business results the client enjoys afterwards.
The method is the "listening lab": a more open-ended version of the
traditional usability test. Listening labs generate strategic
findings - not just tactics - and point the way to measurable
business results - not just usability results like task success or
time-on-task.
In their setup, listening labs are familiar: one-on-one sessions
with a moderator and user, in front of a PC, typically in a lab
facility with a one-way mirror, with observers sitting in the other
room.
Listening labs differ from usability tests in one key way. This is
the one simple thing that, if done properly, can transform the
research and its results. Just four words.
Four Words to Improve User Research:
Don't
define
tasks
beforehand.
When you walk into the testing facility as the moderator, or even as
the client, don't have a script or list of tasks written out
already. Maybe you were told to do so in a usability book, or by a
"guru," or learned it at school. But don't do it.
Instead, when the test starts, interview the customer on how he or
she relates to the website or product in question. (When, and why,
do they typically use this service? What about competitors? What's
the process they go through? What's a specific example from the
recent past, or near future, that they will undertake with this
service?) So, first understand their context: show that you're
listening.
Then, with the customer's context in mind, give them a task to
conduct - the same task they just described to you in the specific
example. As they turn to the PC, ask them to keep talking, describe
what they're thinking, and say why they're taking each action.
At this point, the customer hands you test data you never could get
in a rigid, traditional usability test: tactical learnings,
strategic thinking, and everything in between; almost none of which
you could have guessed beforehand, had you written a script.
Because that's what a task list is, after all - the test moderator
presuming to guess how customers use the service, without even
talking to them first. It's the moderator setting a rigid framework,
saying, "I know how and why customers are here."
But most websites are a strategic representation of the business.
How can you presume to know how customers relate to the business,
unless you ask them first? If your test moderator has a healthy case
of ESP, or can bend spoons without touching them, by all means,
define the tasks beforehand. Otherwise, try the listening lab
method.
And remember that you will get tactical data. Listening labs are
task-based - but only on the tasks most relevant to each user.
Strangely enough, most customers (within each customer segment) will
VOLUNTARILY come up with the same tasks, and the same feedback, on
the site - without any leading from the moderator.
I just finished two days of listening labs for a global Fortune 500
corporation. Creative Good is creating the customer experience
strategy for the entire site, so in the listening labs we had
respondents from all their major customer segments. By the fourth
respondent in each segment, we were seeing the same tasks, the same
feedback, the same results - created voluntarily by the customers
themselves.
Here are some next steps:
- For you: In your next round of customer research, don't define
tasks beforehand. Let the customers lead. (Another good four-word
phrase, come to think of it.)
- For me: In my copious spare time, I'll try to write more about
listening labs, since this is the topic I get asked about most
(along with measuring business results) at the seminars,
conferences, and companies I speak at.
- - -
Get Good Experience by e-mail: update@goodexperience.com
- - -
Come to the conference! Good Experience Live, Friday, April 30, 2004 in New York City.
Back to Good Experience
- - - - - - - - - - - -
Get e-mail updates of goodexperience.com: e-mail update@goodexperience.com
Copyright 1999-2003, Good Experience, Inc.
