Crafting a User Research Plan, Part II

by Mike Kuniavsky

July 3, 2003

The most difficult part of setting up a schedule for your user research plan is integrating it into the existing development system.

In the first part of this series, we talked about setting research goals and putting together a budget. This week, we’ll discuss scheduling, and how research is affected by timing.

Planning Your Timeline

Before you start plotting your research schedule, step back and consider existing schedules and deliverables by asking appropriate stakeholders and project or product managers.

However, don’t let the needs of the existing project drive all of your scheduling decisions. Your research schedule should still be organized so that every research project informs, verifies, and reinforces subsequent ones. In practice, this means that procedures for gathering general information come before those that collect information about specific preferences and behaviors.

Starting Fresh

If you’re lucky enough to start at the beginning of development, your schedule might look like this:

• Internal discovery to identify business requirements and constraints.
• Profiling ideal users and turning those profiles into personas, which are based on knowledge of existing users or the users of competitive products.
• Contextual inquiry to uncover problems users have, both with the product and with the task it’s supposed to aid.
• Task analysis, to specify how users solve problems right now.
• Focus groups, two to three rounds of them, to determine whether people feel the proposed solutions will actually help them and which features are of highest value.
• Competitive focus groups to determine what users of the competition’s products find most valuable, and where those products fail them.
• Usability tests of prototypes that implement the solutions and test their efficacy. Do four or five back-to-back.
• Competitive usability tests to determine strong and weak points in the competitors’ products.

After launch, you can follow this research with:

• Surveys and log-file analysis to compare changes in the product to past behavior.
• Diaries to track long-term behavioral changes.
• Contextual inquiry to study how people are actually using the product.

And then start again. User research works best as a constant process that produces progressively better results.

In the Middle

You may often have to begin your user research plan in the middle of a development cycle. At this point, decisions have already been made about who the users are, what their problems are, and what solutions to use. These cannot be revised — at least not until the next development cycle.

In such cases, your research should begin with procedures that will immediately benefit the product before release. Plan for more fundamental research during the next development cycle. The following order may make sense for you:

• Usability testing in rapid iterations until release.
• Competitive usability testing to understand how well your competitors are doing.
• Log-file analysis before and after release will show you how customers’ behavior changed.
• Survey users immediately after launch to determine the makeup of the user population. Try to understand who’s using the product and how they’re using it.

After you’ve taken these steps, you can begin as if it’s a completely new product with profiling, contextual inquiry, and task analysis.

Starting from the Bottom

A tip about starting your user research at the end of a project: Don’t. Or, I should say, doing user research when a product is almost done and ready to ship is generally a bad idea, and should be avoided at all costs.

You won’t have time to fix anything but the most superficial issues. This is largely a waste of time unless you’re tweaking really terrible problems. Sadly, many people wrongly assume that superficial problems are the only kind that user research (often called “user testing” or “user validation” or “user acceptance”) can address.

If you’re forced to start near the end of a project, there are a couple of things you can do. First stage, and publicize a few rounds of usability testing. Be sure key stakeholders and developers attend, and use the sessions to emphasize the need for a better development process.

Your second option is to start from scratch by pretending the product doesn’t exist, and use the research plan that should have been there from the beginning.

Right on Time

A well-structured research plan is also a communication tool that lets others in your company work with your schedule. It educates your colleagues about the benefits of user research, provides a forum for them to ask questions about their users, and creates an expectation of the kind of knowledge the process can produce.

A sound user-research schedule and process will make all the difference in your end product.

This week’s essay is part two in a series from Mike’s new book, Observing the User Experience: A Practitioner’s Guide to User Research, an indispensable book full of real-world guidelines for how research should be done.