Adaptive Path Blog

Jared Spool. You know him, you love him, and you can see him when he keynotes MX: Managing Experience on March 7th.

He’s graciously allowed us to repost this article so we can give you an idea of what his MX talk is based on.

The 3 Qs for Great Experience Design
By Jared M. Spool
(Originally published: Oct 06, 2008 here)

For more than seven years, we’ve studied how the great user experience teams succeed. We’ve looked at a variety of variables to isolate what it takes. We’ve looked at management structure, employed methodologies, best practices, and hiring qualifications. We’ve looked at team communication techniques, requirement gathering techniques, the target industry, and the geographic location. All said, we’ve inspected about 250 different variables for dozens of organizations across a wide variety of industries, educational institutions, and government.

As with most things, most variables don’t play a role. However, we found three key variables as being critically important: vision, feedback, and culture. Using these three variables, we’ve created corresponding questions to help us quickly rate a team’s experience design prowess. Teams that answer these questions well are far more likely to create great experiences than the rest of the pack.

Factor #1: Understanding the purpose of vision
Here’s the first question we ask: “Does everyone on your team know what the experience will be like interacting with your offerings five years from now?”

When the answer is affirmative, any team member can describe what the user’s experience will be like in five years. They’ll tell us a story, like this real one from a century-old insurance company:

“An insured home and car owner, having just had a tree fall on their garage, will log into the site, explain the damage, upload pictures, and get initial claim approval to start temporary repairs and get a rental car—all within a few minutes. Within the next 24 hours, inspection appointments and a detailed damage assessment are scheduled and reviewed, and the repairs are underway within 48 hours. All the payments are handled electronically from the insurance company, with a single NET-60 bill sent to the policy holder for the deductibles.”

This story is an experience vision. It outlines how the person, in this case someone who insures both their home and car with the company, can make a joint claim and quickly start the recovery process. Notice that the story doesn’t describe the specifics of the design or the system — that’s not important. What’s important is understanding the experience of the policy holder.

While this particular story may not sound that interesting or difficult to someone outside, for this organization it’s a radical departure from today’s experience. Their business units currently don’t talk to each other and pretend that customers don’t exist beyond their own individual products. So, this integrated vision shows a radical departure and eliminates much of the frustration caused by today’s organizational reality. For this organization, five years is aggressive for the substantial, under-the-covers changes that this vision will require.

We like looking five-years ahead because it gets beyond the immediate reactive requirements and starts considering what a great experience could be. If we only looked one year ahead, we’d be stuck with the current realities. If we look too far out, we get into the realm of science fiction.

Because everyone on the team has the same vision, they are all on the same page for what it takes to succeed. Think of it as a stake in the sand on the horizon. Everyone can see the stake and knows when they are taking baby steps towards it and when they are moving away. The stake can move at any time (and, for some organizations, does frequently), but that’s ok, since everyone can see the change and start moving in the new direction.

Struggling teams can’t answer this question affirmatively. They either have never considered beyond the problems of the day or everyone has a different vision. Working to have a solid vision that everyone shares will go a long way to help these teams.

Factor #2: Having a solid feedback mechanism
While the first question deals with where the team is going, the second question deals with where the design has been: “In the last six weeks, have your team members spent at least two hours watching people experience your product or service?”

We’re looking for teams that can answer affirmatively no matter when we ask. That means they are regularly watching the users and learning from them.

These observation sessions can happen in a variety of ways (and in the best organizations, the variety is wide). They can be usability tests or field studies. In each case, each team member has spent a minimum of two hours observing the current experience.

Note that we’re not talking about surveys or satisfaction measures. Those instruments are often flawed and only give a very small piece of the picture. In the best case, they can tell us whether users are frustrated or delighted, but they can’t tell us why. The team needs to observe the experience, in a detailed manner, to really get the information required to make the critical decisions.

Six weeks is an important period. In our research, the average team member works on an experience design project for twenty-four months. This means they’ll encounter a minimum of 16 separate experiences during their tenure, working out to be an average of 48 observations for a four-member team during that period. All of that detailed information can’t help but create better informed decisions in the design process.

Longer than six weeks and the exposure to the users starts to wear off. It’s far less likely that a team member will say, “What about when we saw Fred have problems with accessing multiple policies?” when Fred’s experience happened months before.

Many struggling teams have never had a single member observe the experience of using their design, even though, in some cases, millions of users interact with the design every day. In other cases, they only get data from indirect sources or they’ve had limited exposure during their tenure. When this happens, each member of the team can only talk to their own experience of using the design, which is very likely to be at odds with how real users experience it.

Factor #3: Living a culture that relishes “failure”
The first two questions are straight-forward and make sense, from a strategic perspective. You have to know where you’re going and you have to know what you’ve already built. The last question, on the other hand, can seem counter-intuitive: “In the last six weeks, has your senior management held a celebration of a recently introduced design problem?”

In most organizations, problems are not cause for celebration. However, in a culture that pushes for frequent small changes, problems become opportunities for improvement. Teams that answer affirmatively have established a culture that not only accepts failure, but relishes it as a way to learn about the users and their needs.

At a major software corporation, the CEO regularly holds parties to give out a valued award, shaped as a full-size life preserver, to individuals who have created “learning opportunities” by introducing a problem into the design. Of course, the CEO acknowledges that the problem wasn’t introduced intentionally. But, because it made it into the design, the organization learned important lessons they can use going forward. Receiving the life preserver award from the CEO has become a high honor within the company.

For example, a technology company recently experienced a massive server outage as, upon the release of a highly-desired new feature, millions of users tried to upgrade simultaneously. While the server outage was a major embarrassment (reflected in the press and on Wall Street), it was because of a successful marketing and design campaign for highly-desired functionality. Despite the momentary crisis, the organization simultaneously learned how to create desirable enhancements while also learning the impact that it has on their infrastructure — both valuable lessons they’ll refer to for years to come.

The best organizations hold these celebrations frequently, because they are constantly learning from their mistakes. By making the learning process explicit, through their acknowledgement and reward, the culture starts to look for it. As the old saying goes, “That which is measured gets done and that which is rewarded gets done well.”

Struggling organizations do not hold celebrations of what they perceive to be design problems. Instead, they’ll punish the “culprits” and put new product-preventing policies in place to stop it from re-occurring. Soon, the original stimuli for these policies are forgotten and the organization is doomed to repeat the mistakes.

Driving Towards Improvement
The neat thing about these three questions is their applicability to constant improvement. Teams can self assess and look for opportunities to answer the questions better.

A good team may have a start to the vision, but hasn’t communicated it to everyone who has influence over the design. The team may occasionally get feedback on their current experience, but hasn’t seen anyone recently. And there’s always opportunities to highlight the latest things they’ve learned, even if it was a difficult learning process.

While further research could show there are other factors that influence a team’s success, it’s clear to us that these three factors are critically important. Fortunately, improving them has little downside, making them a serious candidate for any amount of investment the organization can afford.
———–
About MX: Managing Experience
March 7-8 | San Francisco | $1,595
Use promo code BLOG for 10% off
www.mxconference.com

MX is a conference for people who take a leadership role in guiding better experiences into the world. MX serves up examples to learn from and approaches to adopt that can help you lead your organization toward investing in or improving your customer experience on the web, mobile, and more. Over a day and a half we’ll look at the key elements critical to your success as a leader: experience strategy and communicating that strategy, organizational buy-in, results-oriented investment and measurement, and the emerging trends you’ll have to master.

MX 2010 | What is MX? from Adaptive Path on Vimeo.

We hope to see you in March! Register now and use code BLOG for 10% off your registration.

I’m terrifically excited to announce MX 2010, the conference for people who take a leadership role in guiding better experiences into the world. If you need to guide, lead, or drag your organization towards delivering great experiences, this is the event and the group for you.

It’s career-shaping content
We’re ecstatic about our keynote speaker Jared Spool of User Interface Engineering, one of the most enlightening and effective communicators on what it takes to empower design teams and deliver experiences the increase customer satisfaction and loyalty.

We’re also bringing back Margaret Gould Stewart, previously of Google and now at YouTube, she’s arguably the most loved UX team leader and the most refreshing and practical thinker in our practice.

Plus we have Lane Becker of Get Satisfaction, who’s changing the way organizations are delivering customer service in the future. We also have Dan Saffer, popular speaker from Kicker Studios. Peter Merholz of Adaptive Path and Harvard Business Review blogger, will be sharing his most recent discoveries of how organizations make great experience happen. And soon we’ll share more exciting speakers including members of some of the most highly rated customer experience organizations around.

It’s what matters right now
We’re maximizing speakers and content around the topics that make a difference right now to UX leaders. That means we’re addressing head on:

• Creating and communicating an experience strategy
• Organizational buy-in and change to deliver great experiences
• Delivering meaningful business results
• And the emerging trends you need to plan for today

It’s super-concentrated
We’ve been hard at work devising a super-concentrated form of MX, packing into a day-and-a-half more great leaders, more examples of success and failure, and more ideas about how to make things happen. To do it, we’re featuring many short-format sessions and speakers sharing the very best and most essential ideas as focused and fast as they can. For a day-and-a-half of your time, you’ll be leaving with a boat-load of new ideas and new practices for the rest of 2010.

Come make your 2010 great
Come to San Francisco March 7-8 and join our focused community of UX leaders who are sharing and shaping the practices of our field. We have a great low room rate at the event site, the lovely Intercontinental San Francisco. Register now and get in cheap. Do it! And make a big difference in your year for 2010.

I ended up at Adaptive Path through happenstance. A friend through a friend, and suddenly I was working smack in the middle of the user experience design industry for the first time. There were new words to learn, systems to figure out, modes of thinking I’d never come across. And then there were all the other things that made my new environment even more meaningful: relationships developed with big thinkers, conversations about new concepts, and, eventually, the development of my own voice amidst all this smartness.

New environments and situations permeate our lives. We go to new jobs, start relationships, end them, start families, work with new clients…adapting and change are deeply seeded in our everyday. I’ve learned a thing or two about how to adapt to these environments fast – and how to not only survive them, but thrive in them.

On August 12th I’ll be hosting a virtual seminar about methods and tools to adapt, navigate and thrive within the UX field, specifically. This is a great seminar for anyone new to the world of user experience who is grappling with terms and jargon. It’s also valuable for seasoned UX professionals who are looking for ways to increase the impact they have within their organization and the UX community at large.

I’ll be sharing three things that have helped me thrive:

Knowledge Mapping

Learn how to create diagrams that allow you to visualize what you know about the industry, what you don’t know, and the areas where you can supplement what you already know.

Community-Building

Building a community of influential friends is as simple as paying attention to the people around you. I’ll share key things you can do to build authentic, rich relationships with others in the field. And networking is not one of them!

Using Your Own Voice

Whether you’re a student or an experienced UX professional, finding and using your voice will make a difference for your career. I’ll provide you with tools to help you uncover what you have to offer, create a mantra (everyone needs one!) and show a video with tips from four people who have successfully created strong personal brands: Whitney Hess, Merlin Mann, Scott Berkun, and Rachel Hinman.

Join me on Wednesday, August 12(10:00 – 11:15 AM Pacific Time) and uncover concrete steps you can take to help you thrive within the user experience design landscape. Details and registration here.

An interview with Scott Berkun, author and public speaker on
Show Length: 20 minutes

In 1956 a documentary called The Mystery of Picasso was released, showing two hours of Pablo Picasso doing what he did best: making paintings. This film gave the public a first-hand glimpse directly into this infamous artist’s creative process. Public speaker and writer Scott Berkun and I got together for tea to talk about the film and our own experiences around creativity. As both managers of creative teams and creators of work ourselves, we looked at how our processes aligned with Picasso’s…or where we could learn from him. As the discussion unfolded, we came up with an interesting set of guidelines that enable creativity to flourish.

Listen to podcast on www.TeaWithTeresa.com

I just finished working on a project where half of my team was in our Austin office and half was here in San Francisco, which meant 2 different time zones. In addition, our client was in yet another city. Since this was my first time working with such a split team, my learning curve around remote communication, meeting facilitation, and maintaining a strong team dynamic was pretty steep. I thought I’d take a moment to compile those learnings and share.

Tackling Time Zones

• Adjust daily schedule for both offices: A couple of days per week, work on the other team’s schedule/time zone (and vice versa).
• Create a clock for the other time zone on the dashboard of your computer for easy reference.
• People naturally assume you are talking about their time zone, which you may not be. So, make it a habit to always give the time in both times zones when discussing scheduling, no matter what the mode of communication (conversation in the hall, phone calls, IM, email, etc…).

Make It Feel Like Your WHOLE Team Is Right There With You

Use a monitor with dedicated web cam in your project room—sit it at project table (permanently) as if the remote team member(s) is sitting at a spot at the table. It makes them feel a bit more like a real person. (Note: use a Logitec web cam if you can afford it ($100)—quality is much better than iSite webcams.)

Remote Meeting Facilitation

• Adobe Acrobat Connect seems to be a great tool for me thus far. It has video (so you can see the other person/team), document sharing (everyone looks at the same thing at the same time), and the ability to mark on documents (you can point out, specifically, what you are talking about/needs changes).
• During remote meetings, I find that typing live notes about what is said on the shared screen really helps. That way, everyone sees and confirms what was heard and you get double-clarity/reinforcement.
• The downside of Adobe Acrobat Connect: Document resolution is so-so, so don’t use for visual design. Or, send the visual design documents and use the Adobe Acrobat Connect screen simply for reference/orientation during the meeting.

File Sharing

While I don’t like the interface, DropSend.com has biggest file limit out there for the price (it’s free depending on your use).

Shared Calendars

I like creating a shared Google project calendar and embedding it in Basecamp. I give the appropriate people (client team and internal team) the right to add/edit events, meetings, etc. And, you can also make Basecamp milestones feed directly into the calendar.

This is my list of learnings thus far. Please add to this post if you have good tips and tricks to share with the rest of us! And I’d love to hear about tools that you like better than the ones I’ve listed — I’m always on the lookout for better ways of doing things.

In my most recent Harvard Business Online column, I took a departure from my prior discourse, instead meditating a little on being human, and how recent technological development is doing a much better job than organizational development in acknowledging human realities.

From the piece:

…What most excites me about these new means of engagement (which already have amazing successors in university and corporate labs) is that they allow their users to do something that hadn’t been possible five years ago — truly be human. The body is extremely important for human beings, and it’s almost shameful that for so long, such a small part of it was used when working with computers. Mice and keyboards were the product of the Cartesian mentality that mind and body could be separated, that humans could be reduced to brains attached to fingers, eyes, and ears…

…As I’ve been thinking of this technological revolution, I’ve realized we need an organizational revolution. The organizations many of us work in remind me of the state of computer technology from five years ago:

…We’re placed in hierarchical org charts, remnant of railroad and factory operations of the 19th century, and find ourselves in silos that prevent us from collaborating with our colleagues….

…We thus leave the office having only engaged a small part of who we are…

Read the whole thing over there, and I’d love any comments you have.

Part 1 of this interview.
Part 2 of this interview.

Margret Schmidt is among our speakers at MX 2009, taking place 2-3 March in San Francisco. You can register for MX 2009 using the promotional code BLOG and get 10% off. Prices increase January 31st.

PM: We met at a conference last fall where you were speaking about the design and launch of the new TiVo.com website. I believe you mentioned that the site design had not significantly changed for 5 or so years before this most recent launch (and looking at the Internet Archive confirms this. What had been the organizational barriers to change? How were you able to overcome those barriers and launch a radically new design? What did it take to make the site more of an extension of the TiVo product experience?

MS: Historically, tivo.com was treated as an online version of our marketing materials. It was about selling DVRs, and marketing was responsible for that function. Because there wasn’t an interaction design team within marketing, overhauls of the site involved external agencies and lots of money, and didn’t happen that often. As the company evolved the web site did too, and we added product features like online scheduling, and we enhanced customer support tools.

Over the five years where the site didn’t change much, we actually undertook two different redesigns that never launched. They failed for many reasons, but mostly for lack of communication, teamwork, and a shared vision. Different teams had different agendas, and we sent conflicting messages to our agencies.

This last redesign was successful because everyone came together with a common vision. The site as “owned” by marketing, and the redesign project was “lead” by user experience. We had very open communication and shared responsibility. We modeled the project after the way we ship DVRs and features – collaboration and iteration. We did use an agency for vision and high-level design, but also a strong internal team that kept the principles of TiVo’s ease and simplicity in focus during the detailed design and implementation. It was a lot of hard work, but everyone involved knew the end result would be worth it.

Once the redesign was complete, we immediately jumped into the metrics to figure out what needed to be tweaked, and then launched further updates to the home page, “What is TiVo?”, and “Shop” based on what what was working, and what wasn’t. This ongoing work is done internally, with user experience as a service organization working for marketing, product management, or customer support (depending on the site section).

PM: Now to something a little less pleasant. In TiVo’s SEC filings (PDF), it’s recorded that in the last two years, TiVo’s total subscription numbers have gone from 4.4 million to 3.5 million. Obviously, TiVo is in a wickedly competitive market, and, frankly, it’s a testament to the quality of your experience that you’re still around, when what you are competing with is essentially “free”. Still, it must be quite worrisome. As VP of User Experience Research and Design, for what are you and your group held accountable? Do you have any metrics for which you must deliver? What is the charter of the User Experience group in improving the bottom line?

Also, in your seven years at TiVo, what have you had to learn about how businesses operate? How has that changed your view of the role of User Experience in business?

MS: User Experience is responsible for supporting the business needs of various groups. We strive to deliver the best experience for our products, and the best research to inform decision making. We don’t have our own metrics — we share the metrics of our businesses, like selling DVRs or shipping features on a schedule. Over my (nearly eight!) years at TiVo I have had increasing exposure to the business. UE participates actively in product strategy, and shares insights from customers in all aspects of the business, including pricing, packaging, marketing, and support. We bring the customer viewpoint into the conversation, so that the decision maker can weigh it along with the business needs and the technical implications. I think it is critical that businesses have this perspective.

PM: Obviously, I agree that businesses need to have that customer experience perspective, but, clearly, many don’t. As such, I like to use companies with strong UX practices as exemplars. Thinking about that, and the challenges that TiVo is facing, how do you see User Experience maintaining and even improving TiVo’s marketshare or bottom line? What new value opportunities has User Experience identified for TiVo?

MS: It is pretty much the standard stuff. Anything we can do to reduce support costs or increase sales helps the bottom line. When we design features, we think about how to minimize the reasons people might call for support; and we add online self-service tools to tivo.com. To help increase sales, we analyze the reasons people don’t complete a purchase (like they couldn’t tell if the particular model of TiVo DVR would work with the setup they have in their home) and we identify ways to address them. We’re adding a tool to the web site that asks a few questions about your home A/V and networking setup, and then identifies the DVR models that will work for you. We want to give customers confidence in their purchase decision.

PM: I want to wrap up our little conversation here with a look toward TiVo’s future. What new experiences can we look forward to? Thank you for your time!

MS: You’ll see us continue to focus on getting great content to your TV. And we’ll give you new ways to discover the best TV for you – ways that help you get the most out of the channels you already pay for. In these times, when people are cutting back on their entertainment spending, we want TiVo to be a great value for finding and enjoying the TV and movies that are most interesting to you.

Thank you Peter. I enjoyed the interview and look forward to MX 2009!

Earlier this week I got to see what Brian Cronin will be sharing at the end of our Managing Design Projects event, coming up on February 5, 2009, and I got really excited at the end of his presentation. This past year or so I’ve spent some time talking to experts in the field about managing visual design, and sharing ideas about project management. But in my conversation with Brian, he had me commit to something that I’m going to do to expand my practice of program management, and I’ve been bubbling with excitement ever since (yes, I said bubbling). I look forward to sharing with you, on the day of the event, what I’ve committed to, and learning from you all what you’re going to do to expand your practice.

If you’re a program, project, or product manager, come learn about conflict management, facilitation, client relationship management and much much more. Creating a community, and having you all meet each other, is just as important to us as the event itself, so please join us at the beautiful Ft. Mason in San Francisco! Pricing increases to $295 tomorrow, so register today.

Our old colleague Dan Saffer asked a brief question to the IxDA mailing list, “What should interaction designers know about strategy?” It set off an excellent discussion. I responded in the thread, but I wanted to rephrase the post here for the AP Blog.

For starters, there’s a tendency to equate “strategy” with “business”, and I think that can be harmful. There are many examples of design that aren’t suited to business, but none that aren’t suited to strategy.

When I think of strategy in the context of our design work, I think of three things:

• philosophy
• vision
• planning

Philosophy asks, “What are you about? What do you stand for, what is your approach?” Whatever it is that you will be designing needs to be informed by some underlying philosophy, some point of view. The “design mantras” for TiVo that Margret Schmidt shared with Matt Haughey are an excellent example:

* It’s entertainment, stupid.
* It’s TV, stupid.
* It’s video, damnit.
* Everything is smooth and gentle.
* No modality or deep hierarchy.
* Respect the viewer’s privacy.
* It’s a robust appliance, like a TV.

(Margret Schmidt is speaking at MX 2009, by the way. Use RNSB and get 15% off!)

Vision asks, “Where are you headed? How will you know you’re successful?” This vision is an articulation of the philosophy that motivates action. Sometimes the vision can be captured in a single compelling statement, such as Kodak’s classic, “You press the button, we do the rest.” Or the exemplar from bestseller Made To Stick, John Kennedy’s pledge: “I believe that this nation should commit itself to achieving the goal before this decade is out, of landing a man on the moon and returning him safely to the Earth.” A philosophy is insufficient for driving design, particularly something as complex as interaction design. Vision provides the north star that guides your efforts toward a successful outcome.

Planning asks, “How will you get there?” I find that in most discussions of strategy, planning is overlooked, with people more interested in talking about positioning, competition or other big picture items. But when I’ve seen products and projects fail, it’s often because there was bad planning — the go-to-market strategy was flawed, either too ambitious or not ambitious enough, resulting in the release of products that either aren’t yet ready for prime time (say, the first release of Apple Newton) or woefully behind the pack (the first release of Microsoft Zune). Perhaps the single most useful technique we teach at Adaptive Path’s UX Intensive Design Strategy day is the Product Evolution Map, which brings rationality and sensibility to the standard product roadmap.

It’s important to recognize that planning alone is insufficient — most plans have to change, and it’s important that you have a vision and philosophy that keeps the plans on track.

With these three elements in place, your design work is grounded, and set up to succeed.