• Crafting a User Research Plan

    Every piece of user research is part of an ongoing research program,
    even if that program is informal. However, making a program formal
    provides a number of advantages: It gives you a set of goals, a schedule
    that stretches limited user-research resources, and results when they’re
    needed most. It also helps you avoid unnecessary, redundant, or hurried
    research.

    Over the next few weeks, we’ll introduce the three major parts of a user
    research plan:

    • Goals: Why you’re doing the research.
    • Budget: How much it’s going to cost.
    • Schedule: When to do the research.

    This week, we’ll outline the goal-setting process and explain how to
    develop a workable budget for your user research plan.

    Goals

    Before you begin writing a research plan, determine corporate priorities
    and set research goals that can meet them. Hopefully everyone in an
    organization knows the general goals of a project (“Get more revenue,”
    “Bleed less money,” “Get more exposure for the brand,” etc.), but there
    will be a natural spectrum of understanding about what this means. Each
    department team invariably has a different method for figuring out how
    goals apply to them and for measuring their success at completing goals.
    Thus, you’ll need to do some internal discovery before you can make a
    complete research plan.

    Identify Stakeholders and Their Needs

    Stakeholders are the people in a department who represent what that
    department wants and needs from the project. The departments working on
    the project may not be the only ones affected by it, and the people you
    talk to don’t have to be top managers. Seek out groups that are directly
    affected and people who can provide accurate, honest representations of
    how your plans will impact their group. Ask them these three sets of
    questions:

    1. In terms of what you do on a day-to-day basis, what are the goals of
      the product?
    2. Are there ways that it’s not meeting those goals? If so, what are
      they?
    3. Are there questions you want to have answered? If so, what are they?
      If you get answers, how will you use them?

    Stakeholder answers may be contradictory or conflicting, but they will
    give you an idea of what aspects matter most and what questions your
    research needs to address.

    So, for example, say you identify three main needs as stated by
    stakeholders:

    • Help visitors use the search engine better and more often.
    • Tell people about related services of which they may be unaware.
    • Guide visitors to country-specific information from the worldwide
      homepage.

    Now let’s take these needs and prioritize them.

    Prioritize

    If you’re not sure how to prioritize, try the following exercise,
    borrowed from Total Quality Management:

    Make a column next to your list and label it “Importance.” Go down the
    list and rate each item on a scale of one to five, where five means that
    you must have an answer to this question in order for the project to
    succeed and one means it would be nice to have an answer.

    Next, make a second column and label it “Severity.” This will reflect
    how bad the problem is today. Rate each item from one to five again,
    where five represents terrible problems affecting the bottom line right
    now, and one means information that would be good to know.

    Now multiply the two entries in the two columns, and write the result
    next to them in a third column called “Priority.” Ordering the list by
    the third column amplifies the differences between priorities and often
    helps sort items that would otherwise appear similar.

    Next, rewrite the needs as user-specific research questions or
    information to be gathered. Broaden narrow questions into general topics
    to get at the root causes of problems.

    The questions should be simple so that they give the most bang for the
    buck. Pick one or two questions for each need that, when answered, will
    address the underlying goal.

    For example: If visitors need to use the search engine better and more
    often, “How do people navigate the site, especially when they’re looking
    for something specific?”

    To inform people about related services of which they may be unaware,
    “Where do people find related service information? Where do they expect
    it?” or “Where would inclusion of service information be intrusive?”

    To guide visitors to country-specific information from the worldwide
    homepage, “What information is country-specific?” or “Where are non-U.S.
    visitors expecting to find that information? How much is
    language-specific versus country-specific?”

    When you’re doing user research, don’t try to prove things right or
    wrong. Never create goals that seek to justify a position or reinforce a
    perspective. The process should aim to uncover what users really want
    and how they really are, not whether an opinion (yours or a
    stakeholder’s) is correct.

    For more information about setting priorities, read Janice Fraser’s
    essay on the subject.

    Budgets

    The budget will be based on the cost of resources available to you, but
    it’ll probably come in three big chunks: people’s time (including your
    time and the research team’s time), recruiting and incentive costs, and
    equipment costs.

    In my experience, useful times for calculating the duration of a
    qualitative user research project such as a usability test (including
    project management time and typical inefficiencies) are roughly as
    follows:

    • Preparation for a single project — ten hours
    • Recruiting and scheduling — two to three hours per person
    • Contextual inquiry/task analysis — five hours per person
    • Focus groups — three hours per group
    • Usability tests — three hours per participant
    • Analyzing contextual inquiry/task analysis — five hours per person
    • Analyzing focus group results — four hours per group
    • Analyzing usability tests — two hours per person
    • Preparing a report for email delivery — twelve hours
    • Preparing a one-hour presentation — six hours

    The effort necessary for quantitative research, such as surveys and log
    analysis, will vary greatly based on the complexity of the task and the
    tools and expertise available. What a good statistician can do in a
    couple of hours can take days for someone with less training.

    Incentive costs for interviewees vary, but as of Summer 2003 in San
    Francisco they tend to fall around $100 per person per ninety-minute
    session.

    Likewise, equipment costs will vary based on how much documentation and
    space you require. It’s possible to spend $5,000 a day renting a
    usability lab for a series of highly documented usability tests, or
    testing can be free if you conduct a focus group in your conference room
    with a borrowed video camera and use the tape from last years’
    forgettable holiday party.

    Maintenance

    Revise your research plan every time new knowledge comes in. Everything
    will change as your team and the company’s understanding of the user’s
    experience grows. The research goals, especially, should be
    re-evaluated, refined, and rewritten to take every piece of additional
    knowledge into account.

    Since the research you do is likely to affect your understanding of a
    number of different research goals, you should consolidate your
    knowledge about the user experience whenever possible. A good way to do
    this is to create a mini-site for your intranet that contains all the
    reports and goals. Link each goal to the information that applies, and
    enter all problems into the development group’s bug-tracking software.

    Eventually, you should have a set of interlinked documents that,
    together, make up a more-or-less complete picture of your user
    population.



  • Close
    Team Profile