What Adaptive Path Thinks When It Thinks About Eyetracking
by petermeRecently, we had a discussion on an internal mailing list about eyetracking, specifically around why we didn’t use it as a research tool.
Brandon:
First, lack of availability of it and familiarity with it as a research tool.
Second, I find it difficult to interpret the data. So someone did look at something first, second, third, and then ignored some of the rest of the page. I think a good information designer could have devised the flow of the eye on the UI on their own. What I value is the interpretation, which I can get from a few participants “thinking aloud” when walking through a prototype.
Andrew:
I agree that the data may be difficult to interpret, or, at least read into great detail. But, eye tracking can help identify hot spots on the screen or interface that enable the designer to refine the placement of important content or interactions.For example, having the 3rd spot on Google’s Adsense ranking is often desirable. It’s the top spot on the right hand column of ads. Studies have shown that eyes are drawn there more than eyes are drawn to the first and second ad spaces at the top of the page. This affects how some companies buy ads.
Same goes for understanding car dashboards. Knowing where users eyes rest or gravitate towards when faced with continually distracting circumstances helps designers focus on those important locations.
Peter:
Long ago (when I was at Epinions), I looked into eyetracking as a research tool. It was prohibitively expensive. Since that time, I’ve done quite well without it, and haven’t felt the urge to go back. And you rarely hear about eyetracking leading to crucial insights.That said, I just finished watching a presentation that should probably be considered must-see at Adaptive Path — Jensen Harris’ talk at MIX08 on the development of the UI for Office 2007. It’s long (the presentation is 75 minutes, with another 15 for Q&A), and remarkably detailed. Jensen shows many paths considered but not taken, and explains how they got to where they did.
Among the tools Jensen’s team used was eyetracking. When you’re making detailed UI design decisions, eyetracking contributes to some crucial understandings. He showed a movie of someone trying to find the Find feature in Office 2003… It was pretty much all he needed to persuade the Office team that a serious reorganization of the features was necessary.
Todd W:
I did a good bit of eyetracking work a few years ago (even published some papers) and was left with the feeling that it’s not really worth the expense except in very specific situations.Consider my experience: We were evaluating different interfaces for searching video content and wanted to know whether text or still images were most helpful for different tasks. People weren’t good at reporting where they looked, what they spent the most time on, or what was most helpful. Eyetracking helped us figure out that text was still more important to them than a still image — though a small animated clip was most helpful. We spent thousands and thousands of dollars on equipment and many hours getting it to work — these systems are horribly complex and unreliable. I came away feeling that we could have gotten to the place we did much faster and cheaper by just iterating quickly and evaluating all of our concepts with users. While they may not be able to explicitly articulate what they are doing, over time you can figure it out with some trial and error.
The allure of eyetracking was that it gave you statistics — great for academic papers — and cool images of sight paths. It counts as EVIDENCE in a way that other forms of quick and dirty research don’t — sounds like this was the big win for Jensen Harris.
Eyetracking is helpful when you need to know something extremely tactical at a very precise level of detail. But we should think very hard about the payoff. There’s a great deal of overhead and it’s difficult to make this a flexible, nimble process.
Given Todd’s concerns with the cost, I looked around a bit, and found some attempts at using simple webcams for eyetracking. I’d love to know more about such low-cost approaches.
This entry was posted
on Wednesday, March 26th, 2008 at 9:13 am and is filed under Experience design, user research.
You can follow any responses to this entry through the RSS 2.0 feed.
You can skip to the end and leave a response. Pinging is currently not allowed.
This post is licensed under a Creative Commons Attribution 3.0 Unported License.
March 26th, 2008 at 11:27 am
More so than strictly tracking eye movements on an GUI, it would be interesting to track eye movements within a more complete context. It could be valuable to track the eye movements of someone interacting with an car dashboard while they are driving for example. How does limited attention and timing to the interface affect focal points and decision making?
March 26th, 2008 at 12:23 pm
I worked briefly for a firm in Berlin that did traditional usability testing but sort of specialized in eyetracking. They actually did quite a lot of studies that weren’t computer-based: they did studies of print ads, physical store signage, and a couple of other things. They’ve got a section on methods that shows some of the custom tools they’ve built to help clients interpret eyetracking data, and some articles on how they’ve used it in all kinds of studies. Luckily the gear has gotten cheaper and lighter, and really is unobtrusive enough to wear into a supermarket, for example.
March 28th, 2008 at 12:04 am
First a comment to Brandon: Information designers are NOT able to anticipate eye movements, not even for very simple stimulus. As part of my thesis some years ago I did an experiment, where this was evident. Experienced web designers were asked to predict the typical gaze path, and this was compared against the actual eye movement pattern. There were no correlation.
Secondly, eye tracking is often perceived to be expensive, and people believe it is very time consuming, as well as difficult to interpret the data. It is true… if you have to rent or buy an eye tracker for a single study, and don’t have any training in using an eye tracker. Probably you would have to bill customers for rental and the time it takes to learn how to use the equipment. Sure the ROI would be very low for such a study.
But… what if you outsource the study to a company that specializes in eye tracking? A company that got the right equipment, and holds the expertise to interpret the data? What happens when eye tracking studies cost the same as classic UE work? Why should you should not choose eye tracking as a supplement to the classic methods?
March 29th, 2008 at 5:03 am
One of the aspects of eye tracking not mentioned here is it’s great utility while conducting a traditional usability study to provide an additional input stream beyond verbal protocol.
The real time gaze data makes me a much more effective test conductor.
I’ll have to defer on the dialogue on the utility of eyetracking — it’s a much more complicated topic than this forum can do justice to. There are some strong opportunities for quantifying learnability that are otherwise quite elusive.
March 31st, 2008 at 12:03 pm
We’ve done eyetracking several times and we found it very worthwhile especially if you want to track the effectiveness of merchandising on your site. During traditional usability studies people say they hate ads and ignore them, but through eyetracking we can see that they are drawn to them. The vendor we have used, eyetracking.com, does our testing for us and the price is very competitive with doing traditional testing alone. Plus, out executives were absolutely fascinated with the technology and watching the pupils on the screen. Getting and keeping their interest in usability testing is imperative to getting financing.