Adaptive Path Blog

By our talented summer associate Dane Petersen

When you think of a computer, what sort of image pops into your head? Perhaps you see a screen, keyboard and mouse. Maybe you think of a laptop, with its screen, keyboard and touchpad all built into the same unit. If you’re particularly crafty, you might think of a mobile device with a touch screen.

The truth is, none of these objects are actually a computer. They are merely the tools that we use to interact with a computer. We traditionally input data using a keyboard and a pointing device, and the computer in turn outputs information on a screen. A programmable thermostat employs a similar interaction, using buttons for input and an LCD display for feedback. Even on the iPhone, an incredibly innovative mobile computing device, these familiar desktop interactions are still present. Your finger acts as a pointing device, the screen produces visual outputs in response to your input, and you can summon an on-screen keyboard for inputting text.

This desktop paradigm of interaction has been with us for many years, and with the iPhone, Android and Pre it seems to be reproducing well in a mobile setting. However, the “computing” function of a computer, in that it takes in and processes information, requires no particular form factor. As processors get smaller and more powerful, it seems our only limitation is what tools we will use to get information into, and out of, the computer.

Speculation anticipates that the future of computing will involve an ecosystem of mobile devices, touch screen tablets and netbooks, but we need not confine ourselves to this inevitability. To assume that all computer-mediated interactions should be shoehorned into a paradigm that implies a pointing device and screen, is to flagrantly dismiss what is wonderful and unique about our human existence.

Humans are physical beings that exist in a physical world. This “being” is an experience that we all share; a rich, experiential understanding and familiarity with our physical environment. People have an incredible capacity for subtlety and nuance that remains largely untapped in existing digital interactions. Demanding that all our computer-mediated interactions happen through the common screen is an insult to our senses, and an insult to what it means to be a living, thinking, feeling person.

As experience designers it is our duty to celebrate, rather than subvert, the deep, shared experience we have as physical beings. We share a highly-evolved capacity to process an infinite variety of inputs from all of our senses, and are capable of interacting with our surroundings in any number of ways. Our interactions with computers should be no less varied.

Already there is a rich library of interactions in the gaming realm that celebrates the rich, shared tradition of our own physicality. New input mechanisms from Wii Fit to Rock Band all invite unique interactions, and Microsoft’s Project Natal elevates movement-based control to a new level.

These games all involve wonderful physical interactions, but they’ve only scratched the surface of human capability. They break the desktop paradigm in terms of new forms of input, but still depend on a screen for outputting feedback to the player. The use of sound, music and vibrations in gaming helps to enliven the immersive experience, but they remain as supplements to, rather than replacements for, visual feedback.

Moving digital interactions into the physical world can have tremendous value, but developing a compelling experience without the use of a screen can be extremely challenging. In seeking inspiration for these emerging types of analog interactions, it helps to turn an eye to history, to a time long before our interactions were mediated by digital screens.

Based in San Francisco, the Mus√©e Mecanique has an incredible collection of antique coin-operated arcade machines, player pianos and nickelodeons from the late 1800s and early 1900s. That there was tremendous innovation at the turn-of-the-century in these penny arcade games, in step with the tremendous innovation in today’s games, is no coincidence. Entertainment is business, as true then as it is now, and most wonderful things manifest themselves as novelties before they are recognized as something truly useful.

One such device was the Mutoscope, a coin-operated moving picture device built around the turn of the century. As you look through the view port and turn the metal hand crank, it flips through a stack of still photographs like a mechanical flip book. There is something compelling about the materiality of actual physical photographs, and the act of manually animating them is indescribably delightful.

This video features a Marilyn Monroe reel playing on one of the museum’s Mutoscopes. It’s pretty racy by historical standards, in that people are actually kissing one another, but rest assured that it was approved by the New York censors.

This video shows an Englehardt Coin-Operated Orchestrion from 1915. The device reads a perforated paper roll, similar to a punchcard, and translates the bits into notes for each instrument. Even though you are not controlling the device yourself, there is something delightful about real physical instruments controlled by digital punches stored on an analog medium.

The museum has a diverse collection with hundreds of other arcade games, including fortune tellers, music boxes, shooting ranges, car races, and even an animated opium den. While the interactions themselves are quite simple, their experiential qualities are incredibly rich. The coolness of a metal handle, the warmth of a wooden cabinet and the lavish Victorian ornamentation all contribute to an incredibly engaging, tactile experience.

Daniel Rozin’s Wooden Mirror represents a modern interaction that does a wonderful job channeling the delightful, tactile experience of these old analog devices, and giving them new form with invisible computing technology.

The “mirror” is composed of 830 square pieces of wood, each attached to its own motor that controls the angle and brightness of each individual square. A tiny video camera records the viewer, translating their image into pixels that are sent to the mirror in real time. The pieces of wood make a satisfying “clattering” sound as they move into place, further grounding this interaction in the senses.

Expanding on the experiential qualities of this concept, ubiquitous internet connectivity and wireless technology opens all sorts of doors for a distributed system of computers, inputs and outputs. Inspired by the arcade machines of yore, there is limitless potential to augment these traditional analog interactions with networked technology, moving our computing experiences out from behind the screen, and into the world.

This entry was posted on Thursday, September 17th, 2009 at 10:34 pm and is filed under Creativity, Gestural Interfaces, Interaction Design, Mobile, Physical Computing. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.