If you’re using the eenie meenie method to select users for your research, perhaps it’s time you tried something a little more scientific. There is no such thing as sound user research without an airtight user-selection process behind it. No matter how good the observation and analysis, it’s all for naught if you’ve studied the wrong people.
Too much “user research” is conducted, analyzed, and applied without anyone ever having spoken to users. Researchers then offer guidelines based on the needs and preferences of people who would never use the product in question.
Relevant user research results depend on two factors: First, obviously, you’ll need to find people who are likely to use the product. Second, you’ll need to interview enough of them so that trends emerge from their collective behavior. These trends will indicate your primary design targets.
Where to Begin?
Some designers have it easy — for an intranet, they can work with almost any employee; for a customer extranet, they simply call up a few existing customers. But imagine a public Web site with a number of different customer types — perhaps VisitFrisco.com, a fictional site supporting San Francisco tourists.
It ought to support the young and old, the vacationer and the business traveler, the active outdoors person and the sightseer, hell, even the press. We can’t study everyone, so where do we begin?
The first step is to identify key audience types. You may want to ask the marketers how they segment their customer base, though you probably won’t want to use the traditional demographic segmentation.
It’s more effective to identify audiences through behavior. An older vacationer has more in common with a younger vacationer than with an older businessperson. The market research might identify types of vacationers, say, jet-setters, once-a-year travelers, package-tour devotees, and so on. Those distinctions are likely more helpful than age or gender.
You’ll need enough subjects of each type to ensure that you’ll pick up on trends, around four to six per audience. Fewer than that, and your data could too easily be skewed by atypical subjects, more than that and you’ll see the same things over and over, with rapidly diminishing returns on new behavior.
Every project has limits on time and money, so while it might be ideal to study each distinct audience type, we’ll have to choose those that likely provide the greatest return, from both a business and research perspective.
While travel agents comprise a miniscule proportion of those interested in VisitFrisco.com, their ability to influence travel decisions could make studying them a business imperative. Additionally, you might want to focus on once-a-year vacationers because they’re hardest to support. If you design successfully for them, you’ll be serving more active travelers as well.
After making some hard choices, let’s say you’ve decided to study four representatives of these audience types: once-a-year travelers, businesspeople, and travel agents. But we need to get even more specific.
We must find subjects who have traveled recently — either they’ve just finished a trip, or are in the process of planning one. (Obviously, this isn’t an issue for travel agents.) Folks who aren’t in a “travel mode” will be poor subjects, as their responses will be abstract and idealized. Our observations should rely on actual travel experience.
Additionally, you’ll have to find people who live far enough away from San Francisco that travel to the city requires serious planning. However, we don’t need to find people who are planning to or have recently visited San Francisco — VisitFrisco.com will gain from understanding why people choose other cities as well.
To find these particular people, you’ll write a “screener”, a formatted series of up to twenty questions used for screening potential subjects to ensure that you get the right ones. Order the questions from most general to most specific to weed out unlikely candidates as early as possible.
In addition to filtering based on the criteria I just reviewed, pretty much every user research study should exclude:
- people who work in market research and usability;
- people who have taken part in a similar study recently;
- people who are not articulate (good talkers make good user research subjects).
You’ll find a sample screener here (Word .doc, 36k.) Feel free to download it and modify it to suit your needs.
Once you’ve developed a solid screener, you’ll turn it over to someone who can go out and find the right people.
At Adaptive Path, we rarely do this ourselves — recruiting test subjects is an arduous task, involving lots of phone calls, emails, messages, call backs, and so on. We prefer to work with a professional market research recruiter, the kind that typically finds people for focus groups.
Those recruiters have methods for finding just the right people, and they charge anywhere from $100 to $150 a head. For our research, that could mean spending $1,200 to $1,800, but that’s a small price considering the amount of time it takes, particularly with all the other responsibilities we have in our jobs.
Frankly, I don’t feel like I did real user research until I started using recruiters — the notion of finding the people myself was so daunting that I often just didn’t bother.
In our experience, it should take about a week for your recruiter to find users based on the information in your screener. You’ll need to offer the subjects anywhere from $30 to $100 (and occasionally even more), depending on how valuable their time is. Executive VPs cost more than high school students.
If you’ve followed these guidelines, you’re talking to the right people; you have enough of them to discern trends; and your end results will be relevant.
With this foundation in place, you can be confident that the work you do with these users will lead directly to successful results.