Adaptive Path Blog

Hampus Jakobsson presented a fantastic talk at this year’s MEX conference about the “wild west” gold rush mentality surrounding mobile app stores. Hampus warned most players in the mobile space are merely mimicking Apple’s model, leaving many user experience challenges that hinder the app store experience unaddressed. This talk inspired a host of great discussions about many of the fundamental user experience issues that plague app stores and ways to improve the process through design.

However, Hampus’ talk brought focus to a question that’s been lingering on my mind for a while now. As the once innovative app store strategy quickly becomes “hygiene” for many in mobile, I can’t help but wonder if all this fast follower behavior is an incremental step to something much bigger.

What if the real problem with app stores doesn’t stem from Apple’s ridiculous application approval process, scalability problems, or mediocre social recommendation functionality? What if the real problem with app stores is what they are selling?

What if the real problem is the notion of applications on mobile phones?

Applications as a means for both expressing and manipulating information in a mobile context is an interaction model we’ve borrowed wholesale from the PC. While application stores have solved many issues – ease in application development, downloading applications to a device, payment – it’s easy to forget the application model was originally developed for a fundamentally different context. A static context.

What if we haven’t figured out how to accurately express information in a mobile context and we are simply borrowing the wrong model?

I’ve been thinking a lot these days about the notion expression – how artists, engineers and designers have used creative models and methods to express information, points of view, and the possibilities of their time – and moments when breakthroughs around creative expression have occurred.

The web is a great example of inventing new models and methods to express information.

Back in the days of “Web 1.0″ the internet was a vast and unexplored frontier, ripe with untapped potential. While the internet provided an entirely new way for people to access, distribute, and experience information, in 1996 nobody really knew how to create “web experiences” that unlocked that potential.

Legions of print designers applied their knowledge of graphic design and print design to the Internet, giving rise to the phenomenon of brochureware. Some designers applied immersive spatial metaphors to the web, like the famed SouthWest Airlines homepage circa 1996. And who can forget those web sites where pages had the look and feel of pages from a book. Regardless of the model, the strategy was similar; borrowing. We first borrowed models we understood, found our footing and were then able to invent new and more sophisticated ways to express information in a this new context of the web.

Art movements have followed a similar arc. A favorite example was the transition between Medieval and Renaissance Art.

A defining characteristic of Medieval art was it’s lack of dimensionality. Artisans from the Middle Ages hadn’t figured out how to represent form in perspective. Subsequently the work was highly symbolic and representational. It remains valuable and interesting work. However, from an art-making perspective, Medieval art is a study in abstraction. Artisans from the Medieval period lacked the art making methods to represent form in the way humans visually perceive it.

In contrast, Renaissance art celebrated the discovery of perspective techniques such as foreshortening, chiaroscuro and the use of balance and proportion in the art-making process. Artists like Leonardo da Vinci, Michelangelo and Raphael became masters of depicting form in a way that closely mirrored how humans perceive it. Humans were always able to perceive volume and spatial relationships, but it wasn’t until artists of the Renaissance discovered and honed perspective techniques that artwork reflected these qualities.

Data is similar to physical form in that it has perspective. We think about it along lines of place, time, and social dimensions… yet mobile applications rarely allow us to truly experience the multi-dimensional aspects of information. Instead, similar to Medieval art, mobile applications flatten data. Users are forced to either burrow deeply into single application or pogo stick across a host of lightweight applications, often with no through lines for the data. As we begin to prism data through more and more devices – televisions, car dashboards, screens in public spaces – the application model becomes brittle. It locks us into a way of thinking about information that doesn’t accurately represent the multi-dimensional ways we perceive and use it.

What if the app stores and “wild west” application development we’re seeing today in the mobile space is a re-enactment of the evolution of the web? What if mobile applications we download through Apple’s app store are the “brochureware” of what we will experience five years from now? What if applications are a borrowed and broken model we’ll ride out until the “perspective techniques” of data representation and manipulation in a mobile context are discovered and celebrated.

If applications go away, what will replace them? Compelling data visualizations? Adaptive interfaces? I’m not sure, but I’m curious to hear your thoughts…

Mobile is a realm of user experience that has long held my imagination because it’s an accessible opportunity space for designers to explore, prototype and ultimately invent new ways for people to interact with information. Mobile is a place where we can experiment; it’s a place where designers can test the tethers of the PC desktop legacy and create interactions that begin to bring Mark Weiser’s original vision of ubiquitous computing closer to reality.

There have been clear and consistent signals over the last year that indicate the technology landscape is rapidly evolving beyond the boundaries of the PC and mobile devices …

The demo from MIT’s Pattie Maes’ and Pranav Mistry’s wearable Sixth Sense device as well as David Merrill’s Siftables demo were the buzz of TED 2009. These presenters gave the audience of thought leaders insight into the exciting interactions that will be possible in the not-so-distant future.

IBM’s research scientists in India have developed a technology that will offer users the ability to talk to the Web and create ‘voice’ sites using mobile phones.

Barcodes can now hold entire video clips and games with Mobile Multi-Colour Composite, a 2D barcode technology. Better than a QR code, users don’t need internet access to discover associated media—the data is all in the picture.

These signals as well as a host of others indicate we’ve arrived at an important and magical technological inflection point. We’re entering an era – a Golden Age of sorts – that is encouraging interaction designers and user experience professionals to explore the frontier that lies beyond the desktop.

Within this broader trend, I’ll be hosting a discussion on Wednesday, April 8th at Adaptive Path titled, Beyond the Desktop: A Panel Discussion on Emergent Interaction Paradigms. I feel like the luckiest girl in the world to have the opportunity to facilitate a discussion between these thought leaders who are actively exploring this exciting frontier…

• Aza Raskin, head of User Experience at Mozilla Labs will discuss the progress of Ubiquity and represent the promising world of intent-based systems.
• Brent Fitzgerald, and Jeevan Kalanithi of Taco Lab will share their experiences developing Siftables and exploring the realm of physical computing.
• Noah Richardson, manager of Tellme’s Mobile User Experience group, will share his expertise on designing voice-driven systems and interfaces.
• Nathan Moody and Daren David of Stimulant will share their perspective on designing NUI and multi-touch interfaces for the Microsoft Surface Table and other public, multi-user computing installations.
• Jennifer Bove, a Principal at Kicker Studio, will share her perspective and expertise in designing products with gestural interfaces.

I hope you can join us. If you can, please head over to Upcoming and let us know. And if you have ideas about the panel or the topics you’d like covered, comment here or twitter with #btdpanel

If you’re not familiar with Siftables, watch the TED Talk.

Peter Merholz: At TED, you demonstrated Siftables, small “smart” blocks that can interact with one another in interesting ways. What opportunities does Siftables provide in overcoming existing computing paradigms?

David Merrill: Computation used to be in short supply. In the old days of punch-cards, a programmer had to check, double-check, and triple-check their program before they dared to feed it into the machine, because once it ran they were kicked to the back of the queue. If the program had a bug it could be the next day before they got a chance to run their program again. The high cost of bugs meant that experimentation was risky, and programs were written conservatively.

As a result, most people back then had a rather limited view of what computers were good for. They calculated missile trajectories and year-end figures, but the lucky researchers that had enough free access to create video games or embryonic electronic music systems were few and far between.

Today computation has become incredibly plentiful. The netbook I am using to write this essay is so powerful it would have been a classified state secret 30 years ago. The result of our surplus of computation is that we can now use computers in ways that are much more exploratory. Programs can be recompiled in seconds when a bug is discovered. It is often more efficient to whip up a code experiment to try a new idea and see what happens, rather than spending too much time deliberating and double-checking. We now have UNDO.

The observation that has driven my work throughout graduate school, and that continues to drive it forward, is that computers can still be so much more than they are today. Our present-day surplus of computation has been a game-changing advancement towards making the computer a better tool. Looking forward, the major problem is not anymore the amount of computation we have, it’s how we can interact with that computation.

User interface advances can expose new possibilities for how the computer can become a more seamless extension of our minds and bodies. Consider the Nintendo Wii for example. The gestural interaction brought many new users (women, the elderly) into the action, and — equally importantly — created possibilities for gameplay that didn’t even exist before. The graphical user interface (GUI) was a similar (but even larger) revolution in usability that expanded the expressivity of our interaction with the machine, and we are vastly more productive as a result. In both cases the user interface advance tapped into a latent skill that had not yet been utilized by computers; the Wii leverages bodily motion and the GUI leverages spatial memory and visual recognition.

The motivation for Siftables was a realization that there was no human-computer interface that simultaneously leveraged our visual search/pattern-matching capabilities and our manual dexterity for handling collections of objects. Imagine a pile of Legos on the table in front of you, and think about how you would inspect, then sift and sort the pieces in search of particular ones or to categorize them into groups. We skillfully manipulate collections of objects all the time — when we interact with game pieces and playing cards, stacks of photographs, toys such as marbles or toy cars, food bits when we are cooking, and more. These activities involve our eyes (scanning, recognizing) and both hands (grasping, moving), and the vision for Siftables is that they would replicate this type of interaction but for digital content, taking advantage of our existing skills.

Siftables offers a new point in the interaction design space between tangible and graphical user interfaces. It combines elements of both paradigms — physically embodied manipulatives that can be grasped and moved by hand, and screens that can show arbitrary visual information. The inclusion of a screen on each Siftable is a key feature, since it allows interactive roles and content assignments for programs to be visually legible to the user and dynamically assigned at run-time. Tangible systems that use non-graphical blocks or tiles must either assign fixed behavior to each manipulative (which is not as flexible), or project graphics around them (which limits mobility). Siftables attempts to be mobile and physically embodied, while retaining the flexibility of graphical displays.

It is worth noting is that we do not necessarily need to overcome existing paradigms for every task that we do with computers, and Siftables are not a user interface panacea. For instance, I don’t think Siftables are the right tool for writing text documents. For that activity, the keyboard and mouse — or perhaps speech recognition systems — are already doing a pretty good job.

In my opinion, the most exciting step forward that Siftables provides is the possibility for new types of applications and interactions. Think about the millions of flowers that bloomed (software flowers, that is) after the GUI supplanted the command-line. Siftables has the potential to offer a similarly fertile platform for application areas like gaming and entertainment, logistics and scheduling, controlling large-scale complex models, musical performance, education and beyond. We have just scratched the surface with the applications that have been created for Siftables at MIT.

This is a pretty special time in history when electronics and sensing have become so miniaturized, so inexpensive, and so capable. The technology to realize the next generation of interactive tools has arrived, and our task is to invent them. Siftables is a step towards this next generation. I truly believe that as we invent interactive systems in the twenty-first century the limiting factor is no longer the technology, rather it is our own imagination.

Timing is everything. Take Flickr for example, a photo sharing service that successfully emerged in 2004 not just because of good design, technology, and leadership, but because of the coincidental mass adoption of camera phones and affordable high-quality digital cameras. A good idea becomes a great idea if its time has come.

So as we start off in a gloomy looking 2009, I’ll put on my hunch-hat and share my nine ideas of experiences who’s time has come:

		Enabling behavior change — Whether it’s to extend your paycheck or conserve your energy, there’s plenty of reasons for people to change how they behave this year. But behavior change is a complex thing. It’s an experience that needs to be carefully thought through from the human perspective, from the depths of the cognitive psychology of motivation to the breadths of incremental change across weeks and months. People won’t substantively change their behaviors simply because of clever marketing campaigns. To change consumer behaviors we must design motivational experiences that push, pull, and ease the pathway to adopting new habits.
		Feeling the wealth of health — The U.S. stock market dropped almost 40% in 2008, making the phrase, “you always have your health,” more true than ever. Yet investing and participating in your own health and wellness is complex, clinical, and confusing. People and healthcare providers need to engage in simple but sound experiences that foster good decision making, good outcomes, and good feelings. Better experiences that design for the medical, physical, logistical, and emotional experience can make healthcare humane and something we all personally want to invest in.
		Visualizing value — We’ll all be looking to get the most out of a dollar/euro/yaun. The trouble comes when we try to access the true value we’re getting our of a product or service. Experiences that help people find and get the most value out of a product/service will be the winners. The challenges are in revealing the value—especially the non-financial value—and reminding customers of it. Progressive Insurance might help people find value, but few organizations also help people appreciate it the way ZipCar does. When it comes to value, all customers are from Missouri: show-me, show-me, show-me.
		Throwing a party for the third party — Traditional customer-centric product development meant finding customer needs, selecting the most marketable needs to design for, and creating a product to address them. But new approaches can turn this model on its head by opening up organizational capabilities to passionate customers and third party players who can participate in and design solutions that your business wouldn’t or couldn’t consider. Threadless proved this approach interesting. The iPhone app store has proved it real. So what aspects of your experience will you open up to pragmatic third-parties and what experience will you design to support them?
		Uniquely mobile — Mobile is here and it’s been here. What’s changed is we’re no longer trying to shoehorn desktop metaphors and desktop interactions onto mobile devices. What’s changed is that opportunities are opening up for more people to design experiences for mobile devices. As a result, we’ll see many more mobile experiences emerge that are only possible and only compelling on a mobile platform.
		Solid clouds — Cloud computing may be a hot meme, but outside the tech bubble the real world could care less. People will move to the cloud when the experiences offer something tangibly different and better than the desktop. Working more fluidly with a team is one such successful experience, but there will be more. But to find these experiences, we have to pull our heads out of the clouds and find solid on-the-ground benefits to people’s everyday lives.
		Long wow experiences — Yep, I’ll throw in my personal favorite: The long wow is an approach to customer loyalty based on systematically impressing customers again and again rather than simply (and naively) issuing them a loyalty card with an identification number stamped on it. The relationships that customers will keep before, during, and after an economic downturn are the strong relationships with brands that deliver moments of noticeably exceptional service—moments when the service delights, anticipates the needs of, or pleasantly surprises a customer.
		The elegant upgrade — Consumers have already started hanging onto hardware, such as mobile phones, for a longer period of time. This trend could be seen as a positive for the consumer, the environment, and smart business. Software upgrades, add-ons, and other modifications mean new and better experiences for customers. For businesses, it means additional revenue after the original purchase in a positive economy of scale. What has to be created are more elegant customer experiences for upgrading and augmenting products. Such products will be recast as services.
		Chorded services — Multi-channel services typically delivery cacophony, not harmony, across the various channels of customer interaction. The opportunity is to define a songbook of chords that your organization can play as great customer experiences—as noted by Kate in her recent virtual seminar. If businesses start simple and learn to play the chords well, they can coordinate multiple touchpoints to delight customers and support behaviors that results in both savings and positive revenue. Today a business can deliver just about any service over any channel, but by using the lenses of experience you can define what services are valuable when and where.

And here are a few more experiences that didn’t quite make the cut:

• Tween experiences — new businesses that fill in experience gaps between others (e.g.,TripIt)
• Customer servlets — simple service protocols that your business can excel at on specific channels
• Managing personal presence — relating ‘me’ to ‘we’ and ‘where’
• Gaps in personal expression — blogs may have peaked, but there are plenty of other ways to express ourselves
• Markets for talent — if the world is becoming flat, it’ll need more places where people can showcase and sell their talent to pragmatic buyers through a trusted third party

This past week Adaptive Path hosted a few graduate students from CCA, one of our local art and design schools. While it was a challenge getting even a few AP folks to take the time away from their consulting work, we managed to give a tour, share a project room, have an interesting discussion on “what is experience design” and (the best part for me) check out the students’ current projects in progress.

I found their use of Bill Verplank’s framework (recommended by their professors) a useful tool in thinking through their ideas. I had forgotten about this framework and thought it would be worth sharing again. Ambidextrous‘ second issue has a great article on the topic.

The CCA student visit gave soon-to-be-design-professionals an inside glimpse of a design studio and was also fun for AP staff to see what’s up with the next generation of designers. The visit was only about 3 hours of our time, a little coordination with the professor and some quick slide prep (thanks Andrew!). Not a lot to ask and well worth the effort. While we have a robust and active Summer Associate Program at Adaptive Path (AKA internship), this brief visit was another chance to expose more design students to life after college.

From this positive experience, I am posting a challenge to other design studios – internal and consultancies: (if you haven’t already) get in contact with a local design program and set up your own studio tour & student project critique! It doesn’t take much and it’s a win/win for all!

Last night there was a party at Adaptive Path for the completion of Rachel Hinman’s 90 Mobiles in 90 Days blog project.

Rachel started the 90 Mobiles in 90 Days project last June as means of “creative recovery.” There’s more detail on the project site, but the short story is this: Rachel blogged every day for 3 months about mobile ideas, insights, observations and imaginings. Each day, every day. When on the road traveling, when sick, when tired, when inspired. Each day, every day. Finishing this successfully was definite grounds for a party.

The party was fun, with classy sweet nibbles, champagne and interesting people to talk to. We also heard from Rachel about the project: what it meant, how it worked, how it felt to be done. Jesse interviewed Rachel and party guests asked questions. I was inspired by Rachel’s open and articulate comments about the challenges and opportunities of mobile experiences, her thoughts on sustained creativity and the act of making a project.

It’s easy to say “Wow, great project, now it’s done.” But I think really, it’s only started. I see other stages in the life cycle, and we’ve only ended the first.

The beginning: Making The Thing
It was fun to read 90 Mobiles in 90 Days as it blossomed: each day had a new little nugget of thinking, and over time, themes emerged. It was like living vicariously in her brain for 90 days, seeing with her eyes and thinking along with her. Each day was like a sweet little surprise. Some days were better than others. There was a natural ebb and flow of the length of the postings, the images, the completeness of the ideas. Being able to follow along as it unfolded was cool.

The middle: It’s There, But Kinda Dormant
I think the next stage for the site is low-grade activity and dormancy. Thanks to the wonders of Google, new folks will find the site, and additional comments will probably accrue over time. Rachel will most likely keep weeding and gardening, but that’s just bare-bones maintenance.

The stride: Extending the Concept
There are other meta-activities that can happen outside the actual project, now that it’s done. Rachel can extend the experience by speaking about it at meetings, events and conferences. She can write about it on other blogs, in articles, and be interviewed about it. She could write a CHI paper. There’s a lot of space for talking about the thing when the thing is done. Maybe others will take up the meme and do projects for 90 chairs in 90 days, or 90 smiles in 90 days. There is a lot of space for the idea to expand.

The future: The Power of the Time-Capsule
Here’s what I’m looking forward to: What will 90 Mobiles in 90 Days be like in 5 years? When we look back at these ideas, what will these concepts mean to us then? Which ones will we see as harbingers of the new technologies, mobile experiences and devices that we’ll be using then. I think about the videos of future kitchens, in particular the 1967 movie clip from 1999AD and the 1956 Design for Dreaming (kitchen scene starts at timestamp 3:20.) These futuristic concept clips have many elements that seem laughable now, yet there are core ideas that lurk inside them that have become real and part of our daily lives.

So congrats to Rachel, and I’m looking forward to 2013, and seeing how the ideas in 90 Mobiles in 90 Days project foreshadow the changes to come.

The folks at Nokia Design have put together an interesting project: What can you buy for five dollars?

“The global spread of low cost personal communication will have a profound impact on the world around us. It will change our perception of distance and time and affect our notions of community, authority and trust. In some communities lower costs will introduce services such as personal banking for the very first time, whilst in other communities the phone will become an object that is bought and disposed of on a whim. These changes challenge ideas for the future as to what and how we manufacture, and place a greater emphasis on sustainability.

Fivedollarcomparison.org is a small step to broaden the discussion and explore how the impact might vary across cultures and contexts by asking a simple question:
What can you buy for five dollars?”

Here’s how you can participate.

I recently sat down to talk with Raphael Grignani who leads the Nokia Service and UI Design team here in San Francisco. I spoke with Raphael about his involvement in Homegrown, an umbrella project whose goal was to create sustainable, ethical, and desirable communication solutions for Nokia. Raphael shares the journey of this grassroots project that started out as ten inspired individuals within Nokia looking at the topic of sustainability evolved into product and service concepts and eventually found it’s way to Olli-Pekka Kallasvuo, the CEO of Nokia, and the Mobile World Congress 2008 in Barcelona.

Read more…

A recent post by John Kullman on Mobile Crunch highlights a study that speculates by 2011, 25 million Americans will use their mobile phones as mobile wallets. The report states that the ease and convenience of mobile phone commerce is hard to resist.

While I can appreciate the value of mobile wallets, there is something about this prediction that makes me slightly sad.

In a fast-paced and stressful world, who doesn’t need ease and convenience? It’s seems perfectly natural to want to make life a little easier and more efficient. It’s that desire for efficiency that’s inspired countless products and services like automatic bankteller machines, self-serve gas pumps and the revered Octopus and Oyster cards. I’ve used all these systems and services and yes, they have saved me time and made my life easier.

But lately I have been thinking how the systems and services we design in the name of ease and convenience are actually coming at a steep price.

It’s said that people are defined by their relationships to others. Given that, it’s probably easy for most of us to make a laundry list of all the important people in our lives. If one unpacks all the moments of interaction with human beings on any given day, it’s clear a good share of our time is spent reinforcing those important, explicit relationships – a phone call to the parents, conversations with work colleagues, dinner and drinks with friends and family.

The thing is, there are tons of tiny interactions we have throughout the day with people we hardly know – a conversation about organic produce with a clerk at the grocery store, commiserating with folks standing in line at the DMV, a wink from the bus driver while fumbling for bus fare. Somehow those interactions while seemingly less important, have significance in our lives.

They’re important because they give a richness and texture to our daily experience. They add an element of unpredictability and surprise to our lives. They provide us with opportunity to practice skills like striking up a conversation, thinking on our feet, joking, and flirting. Most importantly, I believe the cumulative effect of these interactions feed into the holy grail of human needs – the need to feel connected to the world around us and be part of something bigger than ourselves.

As we clamor to create systems and services like mobile wallets that streamline our lives and make things easier, unbeknownst to us, we’re actually slowly eliminating the opportunity for these types of tacit interactions to occur.

I’m not suggesting that we abolish convenience and efficiency as design principles because that seems crazy. The momentum of the modern world won’t allow it. However, if we simply approach design problems from a task/goal/efficiency perspective, we lose the opportunity to create systems that honor our need for these tacit human interactions and leave in our wake a society of people who enjoy convenience yet feel lonely and disconnected.

How do we change that trajectory? I’m not exactly sure. However, it seems like an interesting place to start would be to think about how mobile technology can “grease the skids” of social interactions. Instead of placing a premium on the accomplishment of a task or a goal, privilege mobile systems that enable the subtlety, elegance and grace of tacit human interaction.

The schedule for UX Week 2008 (August 12-15, San Francisco) is complete, and we think it’s the best conference we’ve put together yet. I thought it would be helpful to relate what I call the narrative of UX Week, as each day is a particular chapter as we go from the aspects of UX practice today, to preparing ourselves for what will come.

On Day 1, we focus on the fundamentals of user experience kicking off with legendary Don Norman, and following that with presentations on the intense redesign of Microsoft Office, and workshops covering a range of essential UX techniques, from prototyping to storytelling to sketching.

On Day 2, we look at the next step in user experience, the emerging field of service design, which aims to address the design of experiences across multiple platforms and touchpoints. Carsharing company Zipcar’s CEO Scott Griffith will explain the wicked design challenges of providing their service, from online reservations to what happens if something goes wrong with the car. We’ll hear from representatives of TheDailyShow.com and Current TV, both pioneers in bridging the television experience to the Web. The workshops on this day will give you tools to succeed in this emerging world, looking towards the future with topics such as information visualization and designing gestural interfaces. We’ll end the day with an inspiring session from Milkshake Media on their work designing the brand experience for Lance Armstrong’s LIVESTRONG foundation.

On Day 3, we immerse ourselves in real-world experiences. We begin with alternate reality game designer Jane McGonigal, who is taking gaming off the screen and onto the streets, and continue with sessions on the institutions that are leading the way with immersive experiences: museums. We then leave the hotel for a field trip to the Exploratorium, perhaps the nation’s premier hands-on science museum, where we’ll immerse ourselves in their lessons on designing complete experiences.

For our final day, Day 4, we look toward the future of user experience. We begin with Michael B. Johnson from Pixar, who will share how the famed studio pulls together its work to consistently create the best animated films on the market. We’ll then look towards designing for the post-PC world, designing for mobile, large-scale multitouch, gestural interfaces, physical computing, and even designing for robots. We will have an expo space for attendees to use Microsoft Surface, ThingM’s WineM, Johnny Lee’s Wiimote Hacks, and other new interaction paradigms. We’ll also look at work we’ve done at Adaptive Path on the future of the Web browser, and finish with design visionary and science-fiction author Bruce Sterling, who will send us out with his rousing message.

We can confidently say that no user experience event points the way forward like UX Week. You’ll walk away with techniques and ideas you can use immediately, and also with a mind for the future of our field.

We’ve extended our discounted registration through June 30. And use the promotional code BLOG and get an additional 10% off! Go to UXWeek.com for all the details on the program.