I’m working with Scott Wayne Indiana on a project that enables kids to use their eyes to control objects on a screen.
Out of the many computer vision techniques we’ve looked at in the Hospitable Room class this semester, eye-tracking appealed to us because it can cater to kids with a wide range of physical mobilities, from mild to severe. Eye-tracking also seems like a more direct route to the brain than any other sort of camera tracking we’ve played with, and could likely allow the user to do things that appear impossible to do – even magical.
In preliminary testing/research of eye-tracking, we’ve learned that any interface we develop most likely cannot depend upon the eye as a reliable cursor. Also, when a user is looking at an object, they tend to lose focus of everything else, to the point where that object is pretty much the only thing they can see.
What compelling activities can result from knowing what someone is looking at – or perhaps what they aren’t looking at? Ideas included laser vision, X-ray vision and more, but we kept returning to two things that we can , both of which lend themselves to play:
- Telekinesis. Moving objects, seemingly with your mind. But really, your eyes would move it. This could be screen-based at first, and then made physical (think magnets under the table, and other “magic” tricks).
- Blowing Stuff Up. Who doesn’t like to blow things up? Staring at an object until it explodes would be fun. This firework show done in Processing looks pretty cool, and this Pixel Explosion example may be a good way to begin playing with exploding existing images.
The destruction of Alderaan is an epic explosion that could be worth trying to replicate in code:
We’ve received some pretty positive feedback on the idea of blowing stuff up, so we’re exploring that idea first. In my mind, here is how it will work: User sees object, then stares at it. As the user stares at the object, the object begins to shake or grow as an indication that something is about to happen, and the shaking/growth gains momentum the longer the user stares at it. At the max time, the object explodes.
At this point, here are some of the most pressing questions:
- Would staring at something until it explodes be compelling to kids?
- What kinds of objects should be blown up? What would kids like to detonate? Should we encourage kids to detonate things?
- What is the best way to make an epic explosion in Processing? Light effects of some sort might be cool.
- Blow up an area of a large image, or blow up a separate smaller image?
In terms of hardware, we’re currently using a baseball cap with a camera and IR LED mounted on it. We will build a pair of eye-tracking glasses from the EyeWriter Instructables, which uses a hacked PS3 Eye camera and a pair of sunglasses to build the hardware for about $50.