Adaptive Path Blog

Part of the Smart.fm iPhone App Story

What if memorizing facts on your mobile device were as easy as listening to your iPod shuffle? It doesn’t demand dedicated attention, you don’t have to worry about what’s up next, and you can tune in and out at any time.

After a day of wearing our mobile hats, we knew this was the kind of experience we wanted to create. Unlike the existing, web-based learning apps, in which you study in 5 to 10 item sessions that have a clear start and end, we want learning on the iPhone to feel continuous. Like radio waves or your iPod shuffle, your learning stream is always playing. You can tune in and answer a question or two while waiting in line, then tune in later and answer a few more. The next-up items are continually downloaded, and there are natural break points between every screen.

But even in a world where learning is journey and not a destination, people still need a sense of progression and achievement.

So as we set out to develop a visual language system for the smart.fm iPhone app, our main question became: How can we communicate progress in an continual learning environment? The flash of insight came when Dan H. likened it to the game experience of Oregon Trail.

While achieving the larger goal of reaching Oregon Trail takes a long time (just like mastering your first 200 Japanese words), Oregon Trail holds your interest by introducing milestones along the way. While you might track your progress towards Oregon in the corner of your eye, you’re generally focused on how far it is til the next milestone.

More complex than Oregon Trail, Smart.fm tracks your learning progress at many levels. In addition to showing your progress towards these milestones, the visual language needed to communicate:

• Item progress (how well you know a fact, from 0 to 100%)
• Goal progress (items you’ve mastered vs. the total number of items needed to achieve a goal)
• The two-part nature of items (like flashcards, they’re composed of a cue and response)
• Your study record (how many items you’ve gotten right or wrong today)
• Milestones (where you’re at now and how far til the next milestone)

Rather than using typical progress meters to communicate these concepts, I set out to develop a coherent visual system that could hold it all together. Thanks to Dane’s Your World concept and Happy Hour conversations with the smart.fm guys, where we chattered about virtual pets and knowledge gardens, I found the answer in a favorite metaphor of mine: The leaf.

I have always loved leaf-themed visualizations. Leaves emerge, mature then fade away if you don’t tend to them — qualities shared by all kinds of personal information, from To Do items, to friendships… to facts you want to memorize!

Through personal explorations and open design sessions — where anyone from Adaptive Path was welcome to sit and sketch with us for an hour including our brilliant “non-practitioners” — we were able to map these qualities to the visual language needs of the smart.fm iPhone app. Here are some visual wireframes showing how it’s coming together:

Item progress is represented by a leaf’s “ripeness” — the more you study an item, the greener it becomes. Once you’ve mastered an item by reaching 100%, the leaf becomes a flower. Throughout the system, green leaves represent items you’ve studied and flowers represent items you’ve mastered.

Goal progress is a visual aggregate of item progress represented by color bars showing items mastered and items studied as a fraction of the total number of items associated with a goal. (By the way, Goals are the new Lists! In the website redesign we worked on with smart.fm, we’re introducing Goals as a new way or organizing learning material around shared motivations rather than topics. More on that soon!)

The two-part nature of items is communicated through the asymmetry of the stylized leaf-shape. The leaf and flower shapes are designed to work both as content-containers and icons. (By the way, this item creation screen also hints at some exciting social features we’re exploring!)

Your study record in the “Now Learning” world is tracked using an Owl and a Beanstalk: For every question you answer, the owl “moves up the beanstalk” leaving behind a green leaf (if you got the question right), a flower (if getting the question right meant you mastered that item), or a bare twig (if you got the item wrong).

Milestones appear every 10 items and give you something to aim for throughout the continual learning experience. Represented by golden eggs, these milestones break open to reveal a progress report, and, if you’ve hit a significant milestone — such as mastering 100, 250, or 500 items — the golden egg might contain an prize! These prizes could be collectible “colorforms” that appear in your little world and provide an ambient visualization of your overall progress in smart.fm.

We are all incredibly excited about this visual system and are already buzzing with ideas about how to take it further in the future (What if the seasons changed every so many items? What if you could arrange your colorforms by dragging them around? Can we still have scratch off answers?).

We’d love to hear your feedback and ideas, and we can’t wait til you can participate in this friendly, social world of learning via iPhone!

Part of the Smart.fm iPhone App Story
By our talented summer associate Dane Petersen

The smart.fm website offers a number of fun learning games that help you master all sorts of world knowledge, from Japanese to French to European birds. Like a sophisticated stack of flash cards, the games learn and adapt to your performance, and constantly tune themselves to providing you with the optimal learning environment. As you progress through the stack, items that you have obviously mastered will automatically appear less frequently than items that you are still trying to learn.

The flash card analogy works well for describing the functionality of the smart.fm learning games, but it provides an impoverished account of their more experiential qualities. In particular, BrainSpeed’s pulsing icons, flapping wings and exploding pufferfish all work together to create a gaming environment that feels face-paced and zany. In designing the look and feel of the learning game portion of the smart.fm iPhone app, we knew we had to create something that would be more engaging than a stack of index cards.

When working on the experience design project for smart.fm, one of our guiding design principles was that the new website would be a fun and open space that invites play. Bringing this concept to the iPhone, we wanted to make the mobile learning game lightweight and playful, easy to start and easy to put away, while still delighting users with fun interactions. People would likely find themselves playing the game during those odd dull minutes of the day, perhaps while waiting for the bus, and we wanted to make sure that these short bursts of play offered a rewarding experience.

In reflecting on these goals, we generated numerous sketches and ideas for ways we could represent the timed, multiple-choice nature of the mobile learning game in a richly experiential manner. We explored metaphors for different ways to show questions and answers, represent time running out, and communicate the user’s progress towards learning an item. We considered the materiality of the game-space, and imagined ways to introduce tangibility through unique interactions. In this video I present a brief walk through my sketchbook, and talk about these explorations:

I took the results of these exploration sessions into a bit more detail, generating a number of sketches that depict potential design directions for the learning game. “Sore” is the Japanese word for “that, that one,” and I oriented this series of sketches around a screen where the user is trying to learn this word, and select its correct response from a series of choices. I talk more about these sketches in the following video:

We thought about an “Advent Calendar” approach, where the user would swipe to open paper doors on multiple-choice items to select their desired response. We also considered a “Scratch-Off” concept, inspired by lottery tickets and scratch-and-sniff stickers, where the user would use their finger to scratch off a response.

Going further afield, we mined the Pogs fad of the 1990s, and cooked up a direction that would involve throwing a “slammer” at an anthropomorphized stack of Pogs in order to select a response. Our interest in Pogs came from a desire to give the user some sort of token as a tangible reward for a correct response. We distilled this concept down into another approach, with Pogs that represent possible responses scattered across a hardwood table. The user would grab the correct response Pog and drag it into a drawer, where they were collecting all of their correct responses.

Finally, we explored a rich metaphor with the natural world, considering a concept where people would interact with a button-based game overlaid on a landscape. As the user answered questions correctly, this world would fill up with small items representing their responses. These items might start as autumn leaves, for example, but as the user answered more questions correctly the world would progress through the seasons, switching to snowflakes, flowers or fireflies. Instead of a conventional timer, the countdown for each individual question would be represented by a rising and setting sun… you’ve run out of time when the moon and stars come out, and another day has passed in your world!

Exploring all possible design directions in these highly generative sessions is an important part of our design process. By keeping the fidelity low and at the sketch level, we are able to entertain a massive number of ideas while still producing a tangible artifact that we can share with other members of the project.

Until recently, I used to bank with Washington Mutual. One of the things that made WaMu unique was their “Occasio” retail style branch design. Those of you who may have seen Don Norman’s conversation with Peter at UXWeek 2008 might recall that WaMu came up with the design after extensive research, field work and ethnography. They spent considerable time and money studying how people do banking and even created some trial banks to help them study customers. The layout, which included a concierge desk, children’s area as well as “teller towers” was friendly, easy to engage with and emphasized more personal interaction between customers and tellers.

I actually used the branch services fairly often. I liked the layout and openness of the branch. But the details stood out. For example, the checking/deposit slips were clearly color coded and easy to read now matter which side was facing out. They were located on an island right next to where customers stood in line, which meant I could fill them out as I waited. When I spoke with a teller, we could both look at the same screen while we discussed our business. It was little things like this that made the customer experience a pleasant one.

Recently JP Morgan Chase took over my friendly neighborhood bank.  The very first change I noticed, besides the new logo on the tellers shirts, was the checking and deposit slips. The color coding disappeared, and the lettering became harder to read because the new slips didn’t fit in the slots the same way as the old ones. Then they were relocated, along with the pens, away from the queue line. Along with the logo, the little things that made my experience easy started to slip away. Soon the free standing teller towers will be a thing of the past.

According Tampa Bay Business Journal, the towers will be replaced with a “traditional” layout, including teller windows behind glass and offices where bankers can sit with customers to discuss products.”

For Chase, The rational behind this move is simple economics. According to Bank Investment Consultant blog  that overhaul is to bring the new branches more in line with Chase’s strategy of aggressively cross-selling financial advice, business banking and mortgage services.

In short, the branches that were originally designed to be customer experienced focused are now being redesigned to better serve the corporations focus.

Now, I understand that in these uncertain economic times, you have to go with what you know. However, it seems to me that the trend for many successful companies lately has been on improving the customer experience. In an industry that has typically been focused on the mission of the corporation, both Jet Blue and Virgin America have been successful differentiating themselves through excellent customer experience in an era of considerable uncertainty. In fact, Jet Blue boasts profits compared to traditional airlines while doing just that.

I recently wrapped up a project with a very traditional company that was focused almost entirely on how to map and improve the customer experience. The result of the work with this company will help with the transformation of its focus on the corporate needs to the service needs of the customer, which will build loyalty and trust between the company and its clients. I expect that more and more companies will turn to improving the customer experience as a way to strengthen and deepen their customer relationships.

I can’t help but wonder if the powers that be at Chase truely evaluated the value of the Occasio design and the impact it had on the customer experience before completely abandoning it?

Hampus Jakobsson presented a fantastic talk at this year’s MEX conference about the “wild west” gold rush mentality surrounding mobile app stores. Hampus warned most players in the mobile space are merely mimicking Apple’s model, leaving many user experience challenges that hinder the app store experience unaddressed. This talk inspired a host of great discussions about many of the fundamental user experience issues that plague app stores and ways to improve the process through design.

However, Hampus’ talk brought focus to a question that’s been lingering on my mind for a while now. As the once innovative app store strategy quickly becomes “hygiene” for many in mobile, I can’t help but wonder if all this fast follower behavior is an incremental step to something much bigger.

What if the real problem with app stores doesn’t stem from Apple’s ridiculous application approval process, scalability problems, or mediocre social recommendation functionality? What if the real problem with app stores is what they are selling?

What if the real problem is the notion of applications on mobile phones?

Applications as a means for both expressing and manipulating information in a mobile context is an interaction model we’ve borrowed wholesale from the PC. While application stores have solved many issues – ease in application development, downloading applications to a device, payment – it’s easy to forget the application model was originally developed for a fundamentally different context. A static context.

What if we haven’t figured out how to accurately express information in a mobile context and we are simply borrowing the wrong model?

I’ve been thinking a lot these days about the notion expression – how artists, engineers and designers have used creative models and methods to express information, points of view, and the possibilities of their time – and moments when breakthroughs around creative expression have occurred.

The web is a great example of inventing new models and methods to express information.

Back in the days of “Web 1.0″ the internet was a vast and unexplored frontier, ripe with untapped potential. While the internet provided an entirely new way for people to access, distribute, and experience information, in 1996 nobody really knew how to create “web experiences” that unlocked that potential.

Legions of print designers applied their knowledge of graphic design and print design to the Internet, giving rise to the phenomenon of brochureware. Some designers applied immersive spatial metaphors to the web, like the famed SouthWest Airlines homepage circa 1996. And who can forget those web sites where pages had the look and feel of pages from a book. Regardless of the model, the strategy was similar; borrowing. We first borrowed models we understood, found our footing and were then able to invent new and more sophisticated ways to express information in a this new context of the web.

Art movements have followed a similar arc. A favorite example was the transition between Medieval and Renaissance Art.

A defining characteristic of Medieval art was it’s lack of dimensionality. Artisans from the Middle Ages hadn’t figured out how to represent form in perspective. Subsequently the work was highly symbolic and representational. It remains valuable and interesting work. However, from an art-making perspective, Medieval art is a study in abstraction. Artisans from the Medieval period lacked the art making methods to represent form in the way humans visually perceive it.

In contrast, Renaissance art celebrated the discovery of perspective techniques such as foreshortening, chiaroscuro and the use of balance and proportion in the art-making process. Artists like Leonardo da Vinci, Michelangelo and Raphael became masters of depicting form in a way that closely mirrored how humans perceive it. Humans were always able to perceive volume and spatial relationships, but it wasn’t until artists of the Renaissance discovered and honed perspective techniques that artwork reflected these qualities.

Data is similar to physical form in that it has perspective. We think about it along lines of place, time, and social dimensions… yet mobile applications rarely allow us to truly experience the multi-dimensional aspects of information. Instead, similar to Medieval art, mobile applications flatten data. Users are forced to either burrow deeply into single application or pogo stick across a host of lightweight applications, often with no through lines for the data. As we begin to prism data through more and more devices – televisions, car dashboards, screens in public spaces – the application model becomes brittle. It locks us into a way of thinking about information that doesn’t accurately represent the multi-dimensional ways we perceive and use it.

What if the app stores and “wild west” application development we’re seeing today in the mobile space is a re-enactment of the evolution of the web? What if mobile applications we download through Apple’s app store are the “brochureware” of what we will experience five years from now? What if applications are a borrowed and broken model we’ll ride out until the “perspective techniques” of data representation and manipulation in a mobile context are discovered and celebrated.

If applications go away, what will replace them? Compelling data visualizations? Adaptive interfaces? I’m not sure, but I’m curious to hear your thoughts…

Next week I’ll be speaking at the 2009 MEX Conference in London. Marek Pawlowski invited me to create a talk around point number three of this year’s mobile user experience manifesto: Customer research methodology must be enhanced to close the reality gap.

My talk – Failures of Imagination: The Role of Research in Creating Compelling Mobile Experiences – was inspired by an NPR segment I caught recently on a road trip in Arizona. Admittedly, talk radio can be mesmerizing and the mind can forge seemingly unrelated relationships between disparate topics amongst the monotony of the desert landscape. However, I believe there was an important message about the role of imagination in the piece Where Were The Media As Wall Street Imploded? that strongly correlates to the primary reason why mobile research often fails in the mobile design and development process. A failure of imagination.

Why didn’t journalists warn us of our impending financial doom? Didn’t they see it coming? Folkenflik’s story gives some compelling and logically sound reasons as to why business and financial journalists were as dumb-founded by the recent collapse of the US financial system as the rest of us. However, it was these words that left my mind a buzz as miles of desert unfolded before me:

“If everything we had said and written came true then we might have expected some of this to have happend. But I think there was a failure of imagination among economists, policymakers, as well as journalists.”

Imagination. It’s not a word we often associate with economists and policymakers… and it’s not a word we easily associate with mobile research, either. These are professional activities that require rigor around the fact finding process. However, the evidence of the impact of facts without foresight is clear. Facts do us little good if we lack the imagination necessary to embrace the futures facts can steer us towards. I’ve seen the impact of a lack imagination with regard to research play out countless times in initiatives I’ve been involved with both inside organizations and as a consultant. Too often research is used to reinforce existing agendas, instead of informing a compelling vision of the future.

At MEX, I’ll share my ideas for how to inject imagination into mobile research and design through defining a clear and informed hypothesis for research studies. I’ll also share three case studies on mobile design research methods developed at Adaptive Path that have proven effective in the creation of compelling mobile experiences.

There’s a stellar line up of speakers planned for the conference, including:
Robert Fabricant, Executive Creative Director, Frog Design
Hampus Jakobsson, Co-founder and Vice President of Business Development, TAT
Sian Townsend, User Experience Researcher, Google

Hope to see you there!

(Sharing my latest podcast…Thought you all might find this one interesting [particularly those into environment design and/or education]…it’s a look at how digital technology is busting open the possibilities in the planetarium field. Enjoy.)

When you think about planetariums, you probably think about craning your head way back looking up at a sea of stars spread across a dome ceiling, right? Well, something is changing in the planetarium world. With the introduction of digital projectors, planetariums are evolving into 3D spaces that are being used for much more than star-gazing. I learned about this revolution through Rachel Connolly, Director of the Rauch Planetarium at the University of Louisville in Kentucky. We had a long distance tea and she told me about all kinds of possibilities for planetarium use in the future and potential impacts upon scientists, astronomy, and education. Imagine if, while learning about molecules, you stood inside one, for example…

Listen to the podcast at www.TeaWithTeresa.com

(And I’d love to hear what you think)

As a systems-thinking, ridiculously rational INTP, I can’t tell you how many times I’ve been told to, “Stop overthinking!” After all, rational thinking isn’t naturally associated with creativity. I admittedly find it difficult to act on creative whim, preferring designs that are the logical outputs of a rational thought-process. To me, a “beautiful” design is one that is logically coherent and rich with meaning.

These left-brained tendencies can be a liability at times: I tend to dig deeply when I should be thinking broadly and can turn a simple idea (or sentence, sorry readers) into a complicated one by logical extension. But in trying to fight these tendencies, I’ve instead discovered ways to channel them into creativity.

I got to try these techniques with others when facilitating a recent workshop for some fellow left-brained people from Smart.fm. I loved these guys — they thought like I thought and didn’t hesitate to bring up rational nits, like “Shouldn’t that one be a blue sticky note? Maybe we should fix it.”

By introducing some “Left-Brained Creativity Methods” (learned from colleagues and conferences) to give structure to our thinking, I found I could channel our analytical energy into thinking experientially and generating out-of-the-box ideas. Here are two of the methods I used during this workshop on understanding and re-imagining Smart.fm’s online learning experience.

Experience Mapping

Concept maps can serve many purposes, but at Adaptive Path, we’ve found them to be useful tools for getting systems thinkers to think from the user’s perspective.

Creating an “Experience Map” involves identifying all of the parts of a system, connecting them, and describing these connections from the user’s perspective. For example, in a typical concept map, we might link the concepts of “tags” and “items” by saying “tags are added to items.” But if you take a moment to ask, “But how does this help the user?” you might get something richer, like “tags help users find and re-find items.”

While seemingly impersonal and abstract, for us left-brained folks, taking a moment to re-think every connection from the user’s perspective helped us get our left brains into the users’ minds.

Metaphor Analysis

Once we’d identified the core concepts and activities in this site’s experience — which included “Sets” and “Collecting” — we took one concept at a time and played a “word association game” where we threw every word we associated with that concept up onto the board. We also brought in random participants from AP for this activity, whose minds were a blank slate (they knew nothing about the site), to join us in free-listing associated words and metaphors. For the concept of “Sets,” for example, we came up with everything from “Bento Boxes” to “Coral Reefs.”

Then, inspired by Chauncey Wilson’s “Metaphor Brainstorming” and Edward De Bono’s “Random Word” methods, we chose random metaphors and deconstructed their characteristics: Games like Marbles and POGs are about “keeping what you win” and have “discrete, tangible, hand-held parts.” We clustered metaphors with similar characteristics together, then used these characteristics to inspire design ideas.

The free association game got us thinking outside the box, while the clustering and analysis activities leveraged our left brains to produce a rich soil from which concepts could sprout.

The team loved these activities, and we produced solid concepts we’ve been able to carry forward. One participant said that brainstorming can be challenging when it’s totally open-ended, and that, for an analytical person, these structured activities and templates made it much easier to generate ideas. More importantly, because the ideas were grounded in rational thinking, he felt more confident that they would carry weight in the long run.

Tapping into left-brained thinking can reveal a powerful force for idea generation. I’m looking forward to discovering more ways to harness the left brain by giving structure to creativity. If you’ve experienced any interesting methods, I’d love to hear about them in the comments!

Will we look back on the desktop experience of today in much the same way we reflect on computer punch cards of yore? If so, when will the desktop and mouse become irrelevant? How do people who want to explore the world of technology experiences that are free from the tethers of the keyboard and mouse begin?

These along with a host of other thought-provoking questions were among the topics of discussion, debate, and jest at last week’s Beyond the Desktop panel discussion. I was honored to be in the company of six brave and talented designers who are exploring the frontier beyond the desktop and thrilled to see such active interest in this topic by the San Francisco UX community.

Here are a few of my favorite quotes of the evening:

We’re still skeptics and I think that is an important perspective to have. I wouldn’t say the technology that we work with is better than anything out there right now, it’s just different. A lot of this is still a hammer looking for a nail. People come to us and say, “we want a multi-touch application.” and we say, “Why?” The challenge for us is developing an understanding for what this technology is well suited for. ~ Daren David

We use play in a lot of our design process. We find as we design stuff, we end up opening a box of things and emulate experiences on the table that way. That is one of the big things that has changed for us – our deliverables have gotten more physical and less visual. ~ Nathan Moody

The truth with all these emergent interactions and interfaces is that the conventions haven’t been established, so you don’t actually know how to work and you end up experimenting a lot more. ~ Noah Richardson

Prototyping used to be a luxury, but these types of emergent interactions, it is an important part of the design process. ~ Daren David

Often times the technology we’re designing for is still being developed. So there’s a lot of give and take and trying to understand what is possible… so we often have to attack from both ends. ~ Jennifer Bove

How do we go from bling to kaching? This is new and shiny right now, but five years from now when this become ubiquitous, what will be the meaningful experiences? And what will be the proper uses of these kinds of technology? ~ Daren David

It really comes down to experimentation. The recognition about a lot of this stuff and the reason I think a lot of people are here is that everybody recognizes and has this feeling that there is potential in this stuff, but we don’t really know what it is.
~ Jeevan Kalanithi

The common element all these interactions share is that they’re all more sociable. ~ Brett Fitzgerald

I have two Roombas in my house and they engage in gladiatorial combat. It’s awesome. I don’t feel like they’re gonna get hurt because they look like frisbees. ~ Nathan Moody

When your Roomba saves your life you won’t feel so cavalier about them. ~ Daren David

… there was a project that reminded us how different emergent interactions can actually open up different affordances and provide accessibility to people who haven’t had it. I have a two-year-old daughter and she instinctively knows how to use my iPhone. It’s frightening. And to see her walk up to the television and try to swipe it… you realize that some of the things being created by natural user interfaces really open things up…. I tend to be fairly optimistic with respect to technology and I think there is this notion of accessibility in a lot of the work that we are doing that we can take a fair amount of pride in. ~ Noah Richardson

I would advise people who want to start exploring interactions beyond the desktop to start by looking at the applications or experiences on the desktop they are currently designing and understanding that it is an instantiation of something that is probably broader. Start thinking about what happens when a user walks away from the computer. What are other the other opportunities? ~ Jennifer Bove

For those of you unable to attend the event, here’s a video of the 90 minute discussion:

Beyond the Desktop Panel Discussion from Adaptive Path on Vimeo.

Sketch note by Kate Rutter

Photo credits:
Panel discussion photo courtesy of Allison McCarthy
Sketch note photo courtesy of Jennifer Bove

Last week, Tim O’Reilly delivered a short address at the Web 2.0 Expo where he offered insight into the five applications he believes point the way for the evolution of the web.

Two themes stood out: sensors will surpass humans in front of their keyboards as the primary data source on the web and Moore’s Law will need to be applied to humanity’s greatest problems. (via ReadWriteWeb)

He cited Google Voice Search on the iPhone, an application that combines both voice and sensor input, as an important technology to watch.

One of our panelists – Noah Richardson, manager of Tellme’s Mobile User Experience group – will share his expertise designing voice-driven systems and interfaces.

He’ll be joined by the following all-star lineup:

I hope you can join us. If you can, please head over to Upcoming and let us know. And if you have ideas about the panel or the topics you’d like covered, comment here or twitter with #btdpanel