If you’re not familiar with Siftables, watch the TED Talk.
Peter Merholz: At TED, you demonstrated Siftables, small “smart” blocks that can interact with one another in interesting ways. What opportunities does Siftables provide in overcoming existing computing paradigms?
David Merrill: Computation used to be in short supply. In the old days of punch-cards, a programmer had to check, double-check, and triple-check their program before they dared to feed it into the machine, because once it ran they were kicked to the back of the queue. If the program had a bug it could be the next day before they got a chance to run their program again. The high cost of bugs meant that experimentation was risky, and programs were written conservatively.
As a result, most people back then had a rather limited view of what computers were good for. They calculated missile trajectories and year-end figures, but the lucky researchers that had enough free access to create video games or embryonic electronic music systems were few and far between.
Today computation has become incredibly plentiful. The netbook I am using to write this essay is so powerful it would have been a classified state secret 30 years ago. The result of our surplus of computation is that we can now use computers in ways that are much more exploratory. Programs can be recompiled in seconds when a bug is discovered. It is often more efficient to whip up a code experiment to try a new idea and see what happens, rather than spending too much time deliberating and double-checking. We now have UNDO.
The observation that has driven my work throughout graduate school, and that continues to drive it forward, is that computers can still be so much more than they are today. Our present-day surplus of computation has been a game-changing advancement towards making the computer a better tool. Looking forward, the major problem is not anymore the amount of computation we have, it’s how we can interact with that computation.
User interface advances can expose new possibilities for how the computer can become a more seamless extension of our minds and bodies. Consider the Nintendo Wii for example. The gestural interaction brought many new users (women, the elderly) into the action, and — equally importantly — created possibilities for gameplay that didn’t even exist before. The graphical user interface (GUI) was a similar (but even larger) revolution in usability that expanded the expressivity of our interaction with the machine, and we are vastly more productive as a result. In both cases the user interface advance tapped into a latent skill that had not yet been utilized by computers; the Wii leverages bodily motion and the GUI leverages spatial memory and visual recognition.
The motivation for Siftables was a realization that there was no human-computer interface that simultaneously leveraged our visual search/pattern-matching capabilities and our manual dexterity for handling collections of objects. Imagine a pile of Legos on the table in front of you, and think about how you would inspect, then sift and sort the pieces in search of particular ones or to categorize them into groups. We skillfully manipulate collections of objects all the time — when we interact with game pieces and playing cards, stacks of photographs, toys such as marbles or toy cars, food bits when we are cooking, and more. These activities involve our eyes (scanning, recognizing) and both hands (grasping, moving), and the vision for Siftables is that they would replicate this type of interaction but for digital content, taking advantage of our existing skills.
Siftables offers a new point in the interaction design space between tangible and graphical user interfaces. It combines elements of both paradigms — physically embodied manipulatives that can be grasped and moved by hand, and screens that can show arbitrary visual information. The inclusion of a screen on each Siftable is a key feature, since it allows interactive roles and content assignments for programs to be visually legible to the user and dynamically assigned at run-time. Tangible systems that use non-graphical blocks or tiles must either assign fixed behavior to each manipulative (which is not as flexible), or project graphics around them (which limits mobility). Siftables attempts to be mobile and physically embodied, while retaining the flexibility of graphical displays.
It is worth noting is that we do not necessarily need to overcome existing paradigms for every task that we do with computers, and Siftables are not a user interface panacea. For instance, I don’t think Siftables are the right tool for writing text documents. For that activity, the keyboard and mouse — or perhaps speech recognition systems — are already doing a pretty good job.
In my opinion, the most exciting step forward that Siftables provides is the possibility for new types of applications and interactions. Think about the millions of flowers that bloomed (software flowers, that is) after the GUI supplanted the command-line. Siftables has the potential to offer a similarly fertile platform for application areas like gaming and entertainment, logistics and scheduling, controlling large-scale complex models, musical performance, education and beyond. We have just scratched the surface with the applications that have been created for Siftables at MIT.
This is a pretty special time in history when electronics and sensing have become so miniaturized, so inexpensive, and so capable. The technology to realize the next generation of interactive tools has arrived, and our task is to invent them. Siftables is a step towards this next generation. I truly believe that as we invent interactive systems in the twenty-first century the limiting factor is no longer the technology, rather it is our own imagination.
On Day 4, we’ll have David Merrill, one of the creators of Siftables. Read his response in my 
Entries(RSS)