Adaptive Path Blog

Teresa Brazen & Julia Houck-Whitaker

When we set out to make Aurora we thought, “This can’t be that hard. We’ve made concept videos before.” But the complexity and ambiguous nature of the production process caught us by surprise, challenging us far beyond our expectations. From a hospitalized actress to a managing almost 1,000 art assets, this blog post will share some of the unique challenges we stumbled upon.

Casting

Usually, concept videos include wealthy white people in their mid-twenties. Our casting goal was to break that stereotype. The browser is an application for everyone, and we wanted to reflect that universality. Given limited budget and time, our first inclination was to ask friends and family to be our actors. But, with nine diverse actors to cast for three full days of shooting, we quickly realized we had to reach outside of our circle.

Craigslist proved to be an invaluable resource. In two weeks time, we held multiple casting calls, hired nine actors, and found our photographer, JP Dobrin. Working with professional actors, rather than friends, proved to be the smart choice, which is reflected in the acting and imagery in the final deliverable. As with any complex project, the unexpected happened during one shoot: an actress was hospitalized and we had to make last minute changes to our cast. Rebecca Blood (Jesse James Garrett’s wife and, luckily for us, an experienced actress) stepped in to play the role of Moira last minute. This experience reminded us that movie making truly is the art of creative problem-solving.

Location Scouting

The last minute juggling wasn’t quite over: the day before the first shoot, the toy store we planned to use backed out. Julia, the project manager, made a flurry of quick calls and found a store manager who graciously offered her store, Ambassador Toys. We didn’t get a chance to visit the store before the shoot, but we lucked out: the store’s bright colors and skylights made the setting come to life. Given the short window of time for shooting all of the scenes, we used one house (thanks to our HR Director, Jennifer Bolduc) to shoot two scenes: a farm and a suburban home. The trick was to alter the aesthetic of each room so that they appeared to be entirely different locations. Jennifer’s gardening gloves and boots made the farm setting more realistic, and she graciously let us move her furniture around to transform her living room into a suburban setting.

Props

In concept videos, props can easily appear unrealistic and distract from the narrative. We recognized that risk and strategically chose to limit the number of props while keeping those that we did use very simple. In fact, none of the characters use a mouse in any of the scenes. The mobile devices are simply thin plastic rectangles with green paper adhered to their surfaces awaiting animation. The screen in the living room scene is also a large green piece of paper made to look like it is part of the wall. We realized the less props we used to predict what the technology of the future looked like (besides the user interface of the browser) the more believable the final product.

Planning & Preparing for the Shoots

Given the limited time to shoot the video, it was vital that we had a thorough plan to guide our on-set efficiency. We utilized storyboards and one-page shoot summaries for each day of shooting to help us develop a detailed plan. We created the storyboards in Omnigraffle that captured three key components:

* A basic composition sketch of the shot
* Dialog or animation occurring
* Notes for photographer and animators

The Shoot Summaries briefly outlined and tracked every aspect of the production, such as setup and shooting times, when actors arrived, clothing/makeup arrangements, equipment and props needed, etc. We also met with Whiskytree, the animators/editors of the video, multiple times prior to the shoot to ensure that we fully understood the deliverables we were expected to produce. (Special thanks to Whiskytree for their patience with our asset production and remarkable turnaround time.)

Communication

That said, communication proved to be one of the most challenging, time-intensive, yet vital aspects of production, both in-house and with the animators. The visual design assets alone created for this video numbered close to 1,000. We also had to approach visual assets in an entirely new way. Usually, we create static visual compositions of web pages or screens, but in this case, the visual assets were animated — their sizes and positions changed. In fact, we sometimes discovered our ideas simply wouldn’t be possible in production and had to create alternatives. The variables were moving — literally. This meant that the details in all the visual assets were extremely important.

We found that the storyboards proved to be highly effective in helping the animators visualize abstract concepts, as well as giving the photographer direction during the shoot. Jesse James Garrett also created videos of himself acting out the actions in particularly complex animations, to give Whiskey Tree another visual communication tool. Face-to-face meetings internally with Whiskytree ensured we were all on the same page. While producing each asset took longer than we expected, we had detailed spreadsheets that tracked the status of each one that we used to communicate with the animation studio.

Directing

We discovered that we needed four people on set:


1. Director to guide the actors and discuss shot composition with the photographer
2. Assistant Director to watch out for continuity issues and maintain a “big picture” perspective
3. Photographer
4. Production Manager to handle logistics on the set (food being a vital logistic)


In the first few hours on the first day of shooting, we quickly realized the importance of having clearly defined responsibilities. Initially, the lack of clarity around roles slowed production down, but once resolved, we shot higher quality footage in less time. In fact, we budgeted three hours of shooting time for every two minutes of video, and we used all of it.


Ignorance is Bliss

On one hand this rang true for us, on the other, it made for long days and early mornings. Given that we didn’t fully understand what we were undertaking at the onset, we were neck-deep before we realized just how ambitious we were. As a result, we completed a very complex project that we might not have if we understood the implications of managing nine actors, four design firms, a photographer, and an animation studio. By the end, we were both challenged and exhausted by an intensely satisfying learning curve.

We didn’t collaborate with anyone from Google on the Aurora project to envision the future of the web, so we were especially interested to see their new browser, Google Chrome, and the ways in which it echoes some of the same ideas we came up with for Aurora. For a tour of the more interesting user interface touches in Chrome, check out pages 18-24 of Google’s comic book about the product.

As we explored how people use tabs to manage the web experience, it became clear to us that the notion of the “browser window” as the top-level component of the experience made less and less sense. In Aurora, we completely did away with windows, having all the interface elements that are usually attached to them reside in the frame or radial menus, all of which would typically be off-screen. Chrome doesn’t go quite that far, but by decoupling tabs and windows and allowing dedicated chromeless windows for web applications, it does deliver a more componentized approach embodying some of the same principles.

Both Aurora and Chrome see the browser’s location bar evolving into something much more flexible and useful. In Firefox 3, the location bar became the “awesome bar”, enabling users to search any part of a URL, page title, or tag, in both the browser’s history and bookmarks. In Aurora, we envisioned the awesome bar becoming the “way awesome bar”, a general text interface to everything in the browser as well as to services on the web. Google Chrome takes a step in this direction with the “omnibox”, which adds full-text search of browser history and access to web search services. (Mozilla is experimenting with moving toward the way awesome bar as well, with a project called Ubiquity.)

Chrome’s “new tab page” touches on one of the most significant themes in the Aurora concept: the idea that the browser learns from user behavior. The new tab page in Chrome presents the most frequently accessed pages and search services for that user. Aurora’s spatial view gives visual emphasis to objects based on frequency of use as well, but adds into the mix an analysis of user behavior patterns so that you’re not just seeing the same nine sites every time, but you’re seeing a different mix of sites depending on where you are and what you’ve most recently been doing.

By the way, there are a lot of interesting ideas for evolving the browser being discussed in the Mozilla Labs Concept Series forum. Check ‘em out!

For most casual web users today, the smallest thing on the Internet is a web page. This is the element that a user can recall via a URL, store in a bookmark or forward to another user. Sure, there’s sub-elements of a page such as the text or an image or an embedded video, but most users are not savvy enough to deal directly with these. More importantly, users typically don’t care to break sub-elements out of the context of their web page since the browser offers no compelling reason to do so.

For Aurora we saw an opportunity to change the browser’s focus from the page to the individual “object”. Giving users the ability to interact directly with the atomic parts of a given web page offered greater personalization of their Internet experience. As we see in all three of the movie segments users can tear off parts of a web page and store them for use later in their browsing. Sometimes those parts are recognized as people objects and sometimes they are discrete data objects. The objects can be combined with other web pages or other objects. For example, refer to the second segment where a calendar event (data object) is pulled from a web page and dropped onto a person that’s resting in the browser frame.

My favorite combining of objects is displayed in the first movie segment where rainfall data is styled to be more usable. The browser automatically recognizes that the data bound to the page is interactive and this data reacts visually when a second presentation object is dragged on top. The intersection of the data provided by the National Weather Service site and the preferred presentation from the user creates an information source that clarifies the rainfall trends for our actor.

At UX Week I was thrilled to see Jeff Veen present a similar idea in his session titled “Designing Our Way through Data“. Amazingly his example even uses rainfall over time as the data source. Here’s the five relevant slides (pdf) from his presentation:

During our brainstorming we envisioned that a marketplace could open supporting the creation of presentation objects. You don’t like the way that Google search results look but you like the underlying data? Well, just purchase Dan’s Fancy Results (only $9.95 for a limited time) and add it your browser. Did you create an amazing mash-up with data from Amazon and Last.FM plus your own visual design? Sell it in the presentation object marketplace!

Like many of the ideas in Aurora, objects are based on the extrapolation of current technologies. The semantic web is already recognizing and defining the atomic elements of a web page. The browser can sniff out these elements (microformats) via plugins like Operator. By default all modern browsers also present the existence of an embedded RSS feed to the user. Our notion is that data and presentation objects in Aurora are some evolution of markup, like XML.

A prototype was released by Mozilla Labs this week that tackles many of the same concepts. Aza Raskin describes the Ubiquity concept as “allowing everyone–not just Web developers–to remix the Web so it fits their needs”. Similar to Aurora’s data objects, this experiment relies on the ability of the browser to see information within a page as discrete elements. The atomic level of the web is reduced to something smaller than the page.

Ubiquity for Firefox from Aza Raskin on Vimeo.

While we were designing Aurora, we thought about how the experience would scale and translate across multiple devices. For the mobile scenarios, we had to design something that would work on a multi-touch enabled mobile device.

As of today, almost all browser experiences are designed without considering the mobile user. Instead they are derived purely from the desktop experience. My colleague Rachel Hinman says: “I still believe most Internet experiences on mobile devices are broken and compromised, overburdened by interaction models and metaphors from the PC that simply don’t work on small devices. Yet so much of how we understand the Internet – and computing – is based on the PC legacy.”

I was given the task to translate Kumi’s design directions for Aurora from a desktop experience to a mobile experience for the video. My key concerns were to maintain functionality, consistency and respect for the constraints faced by a mobile user.

Since we wanted to design as realistically as possible, we had a prototype made in plastic that reflected the exact form-factor of the mobile device.

Prototype made in plastic (front)

Prototype made in plastic (side)

While iterating various designs on how Aurora would translate for the mobile experience, I always explored how it would look on the actual device by attaching a design print out on top of our plastic prototype. This enabled me to study the sizes and placement of the elements to provide an experience that remained consistent with both of the other Aurora scenarios: the desktop as well as the large-scale, gestural interface. Continue reading this post by Jesse that explores the design themes that drove the three scenarios.

Writing the script for Aurora turned out to be a tricky balancing act. We wanted to illustrate the interesting design solutions we had come up with, but we also had to provide enough context to make the solutions meaningful. We needed the movie to have a narrative flow and momentum, but it also had to cover a diverse array of interactions. And just to give ourselves an extra challenge, we set for ourselves the goal of avoiding what we considered to be the clichés of the design concept video: affluent mid-thirties professionals in sleek modern environments, or chic young urbanites out on the town.

As a result, we ended up spending a lot more time working on the script than we expected. Dan Harrelson, Julia Houck-Whitaker, and I went through several iterations of sticky-note exercises: first prioritizing the interactions we wanted to illustrate, then brainstorming user tasks that would involve those interactions, then trying to stitch the tasks together into plausible scenes.

We decided pretty early on that it was impractical to work all of these interactions into a single overarching story. Instead, we settled on three segments: one focused on using a desktop computer in a work context, one focused on using a mobile device in a social context, and one focused on using a large-screen home device in a family context.

Of these three, the mobile segment turned out to be far and away the most difficult. We developed and discarded idea after idea as we realized that each one focused too much on the functionality of the device, and not enough on the functionality of the browser. It was fairly late in the scriptwriting process, after we thought we had all the scenes plotted out, when I decided we really needed two mobile sequences: one oriented around location-aware services, and one around interaction between the web and the physical environment.

When we did our first timed read through of the script, we were dismayed to discover that it came in at nearly double our target length of six minutes. (The film industry has a rule of thumb that says a page of script equates to about a minute of screen time; this didn’t apply to us because our script contained long, complex descriptions of user interface behavior that would take just a few seconds to unfold.)

So I cut it down. And cut it some more. And cut it some more. Much of what I cut was dialogue, intended to provide a bit more context on what the characters were doing and to hint at some of the technological changes suggested by the scenarios Jamais wrote. Finally, I decided I’d cut too much. I went back and rewrote the script from beginning to end, reworking it so that certain lower-priority scenes could be included if we had time to do them (as a possible “extended cut” of the movie) but ensuring that the narrative flow didn’t depend on their presence. (Only one of these, the exchange between Harry and Beth about MapQuest, found its way into the final movie, which still ended up over nine minutes long.)

Scrivener was an enormously valuable tool in the scriptwriting process. I didn’t use all of its functionality, but it did provide vital, specialized tools above and beyond what a word processor can do. Next time, we’re eager to try out Celtx, for the functionality it provides to bridge the gap between writing and production.

As written, the script doesn’t accurately reflect what ended up on screen. Many of the interactions I described in the script just didn’t work when I saw rough versions of the animations produced by Whiskytree, and in a couple of places I had to rethink the interaction flow very late in production in order to make the interface consistent and realistic.

Download:
Aurora Script

Looking back at the seminal web browser Mosaic as we started visual design for the Aurora project, I was surprised to find that our present browsers haven’t changed much in the last decade. As you can see, the general functionalities are constructed with similar items such as URL field, bookmarking, back/forward button.

With Aurora, we envisioned a browser that provides a less static, more natural interaction — a direct and intuitive way to interact within the web, similar to the way we relate to objects in the real world. We came up with the following key concepts for the visual design of Aurora:

The visual aesthetic should convey lightness and simplicity. The spatial layout should feel clean, open, with an expansive three-dimensional space. The spatial view reflects events unfolding in real time. Archived items are presented at a distance while current items appear immediately in front of you, suggesting time and space. Objects invite you to grab them and are governed by real world physics with responsive feedback. Everything you see in the space is touchable. You can grab, drag, and drop in three-dimensional space. Objects are floating subtly in real-time and are seemingly influenced by gravity as well as other objects and time. Object groups are clustered by relationship and exhibit motion behavior reflective of their association. Colors are utilized as cues to different behaviors — highlights of different colors denote notification, clustering, drop zone. Shapes are used to differentiate objects such as people, places, and data.

Process

We began with the mood board below to capture the emotional characteristics of the browser’s look & feel. These images from the mood board reflect visual ideas that were explored: bubbles, dew: soft and organic yet structured ethereal entities.

We then explored shapes that felt playful and expressive. We created shapes that are associated with the function of things.

Explorations on the wheel navigation element:

Explorations of the spatial view and navigational look & feel:

This video is best viewed in high definition. To view at 1280×720, press play, then click the HD button, then zoom to full screen and turn scaling off.

Here’s the complete Aurora concept video containing only the scenes of user interaction and leaving out the commentary segments. This version has some additional material connecting the four scenes together into a single narrative.

Video segments with commentary: Part 1 | Part 2 | Part 3 | Part 4

A few more notes on some of the feedback we’ve received about Aurora:

• Is Aurora a browser that takes on some of the functionality of the desktop? Or is Aurora a desktop environment that integrates access to resources on the Web? The answer to both questions is yes.
• The moments where the system doesn’t quite work for people are there on purpose. In the future, technology is still imperfect.
• We couldn’t tackle as many aspects of the Web as we would have liked. If we left out something that you think is important, it’s not because we thought it wasn’t. We had to set priorities, and we did that by focusing on the areas outlined in our design themes.
• Remember, if you disagree with any of the choices we’ve made, propose your alternative solutions in the Mozilla Labs forums.
• A few people spotted the G-Man, but did you notice the screenshot of Netscape 0.9? Or the agent from Knowledge Navigator?

This video is best viewed in high definition. To view at 1280×720, press play, then click the HD button, then zoom to full screen and turn scaling off.

In the conclusion of Aurora, the browser goes home, moving to a large-scale, gestural interface.

Watch: Part 1 | Part 2 | Part 3

Credits for Part 4

Written and Directed by Jesse James Garrett
Producer: Julia Houck-Whitaker
Assistant Director: Teresa Brazen

Photography: Jean-Philippe Dobrin
Animation and Video Production by Whiskytree

Browser User Experience
Lead Designer: Jesse James Garrett
Design and Technology Advisor: Dan Harrelson
Visual Design: Kumi Akiyoshi and Sebastian Heycke
Production Support: Judd Morgenstern and Lin Lin

Web Page Design
Amazon Workspace: Chris Glass

Cast
Tim: Kamasu Livingston
Kelly: Noah Haydon
Samantha: Daphne O’Neal

A foundational concept within Adaptive Path’s Aurora browser vision is that your web is always available. This idea is visualized in the video when the New York Times’ article Jill is reading on her desktop computer later appears on her mobile device.

What do we mean by your web? With advances in computational power and limitless storage, we envision a future where everything you do online is recorded and available via any device. Your web includes your entire web history and all of your saved objects, be they people, data, sites, etc. All of this information is saved to the “cloud” where it is now available to any other computer connected to the Internet.

Combining months, years, or even decades of your personal history with the context of a device opens the possibility for real insights that help a user make the most of their surroundings. Take for example the personal route that MapQuest recommends based on the users’ interest in bike shops and a book title.

Simply authenticating with a new device provides users with instant access to the last state of their browser. The user interface will appropriately render your web to fit the device, everything from tiny screen on a watch to a wall-sized touch-screen. A critical concept is that while most users will have personal devices on which they interface with the browser, your web can be accessed via any device. Walk up to a public computer in an airport or a library, authenticate yourself, and the browser immediately has access to a user’s lifetime of web browsing. This would also work on other people’s devices, such as a friend’s mobile or television.

We envision that the vast amount of personal information available to the browser will create powerful trend and recommendation engines that applications can utilize for future needs. For example, a decision that a user made months ago on a home computer can instantly inform a travel planning web app accessed on a mobile device used at an international airport.

It’s your web and it’s always up-to-date and always available, wherever you are.