New device provides a virtual sound studio
CAMBRIDGE -- James Patten is standing in a cramped room filled with wires and electronic components, looking for a good beat.
In front of him is a strange device he calls the AudioPad. It is a white board, just a few feet wide. Resting on the board are several disks that look like miniature hockey pucks, each embedded with a tiny device that allows a computer to track it. The entire surface is lit up from above with images from a projector.
Every time he slides one of the pucks across the board, the electronic music that issues from a pair of speakers shifts in response: a high line of percussion fades in, a looping series of tones changes from one set to another, an old-school hip-hop beat morphs into a frenetic breakbeat.
Patten has invented a new way to create electronic music. AudioPad is a form of virtual reality, where plastic pucks you push with a finger interact with a virtual world being projected onto the board. In the case of electronic music, the pucks can move controls for the volume or the tempo of the music, for example. Patten and his collaborator Ben Recht, both graduate students at MIT's Media Laboratory, have already performed around Cambridge and done a few gigs in Europe.
But AudioPad also represents a new way of interacting with a computer -- an "interface" in the jargon of engineers -- which could have applications far from the clubs where people come for strobe lights and electronic music. The computer could just as easily conjure up a model of a city's transportation system and allow the user to push around a bus or train line, using one of the computer tracked pucks, to see whether the traffic flow improves.
"Music is a really demanding application because things have to work well, and in a way that makes sense to you while you are performing in front of people," Patten said. "This can really push the design of the interface, and later you can apply [what you learn] to other types of problems."
The work is part of the increasing interest in finding more creative and efficient ways for people and computers to interact. In the beginning there was the punch card, which was used to input information into early computers. Then there was the keyboard, which has been replaced by the keyboard and mouse and all the drop-down menus today's office worker takes for granted. Today there are new approaches, like controllers that give touch feedback, massive virtual reality rooms, and, now, AudioPad.
"This is a neat system," said Ken Hinckley, who was not involved in the work and is a research scientist at Microsoft Research in Redmond, Wash. There is a general recognition that today's computer interfaces don't take full advantage of all the skills people have, and AudioPad is part of an effort to change that.
"Imagine someone sculpting clay, using all ten fingers, and contrast that with what you do with a pointing device," said Hinckley. "There is a tremendous difference there."
Patten said that he has had inquiries from companies that are interested in turning AudioPad into a commercial product. The basic technology, an interactive table that is able to sense the location of objects on it using radio waves, has been licensed to a Japanese company, he said.
It is difficult to explain how AudioPad is used in words, the same way it is difficult to explain to someone how to cut and paste text on a computer instead of just showing them. Patten, who works in the Tangible Media Group at Media Lab, recently gave a Globe reporter a demonstration.
First, Patten slid a puck over a box, projected onto the board by the computer, labeled "Bass." The moment the computer saw the puck entering the box -- it constantly tracks each puck -- the word "Bass" was projected next to the puck, and it followed the puck as he dragged it back to the center of the board.
Then he dragged a special star-shaped puck, called the "action puck," next to the first puck. A menu of options, each a different bass line, seemed to spring from the first puck, projected onto the board, and he dragged the action puck to the one he wanted and the bass line started to play from the speakers. A violet arc of light began to rotate around it, indicating it was playing.
Using other pucks, he started a percussion line and other musical tracks. He brought in another bass line, substituting it for the first one, then alternating between the two. Soon, a thumping electronic composition was coming together. Patten was caught up in the music, his left knee rocking slightly. He had found his beat.
Gareth Cook can be reached at cook@globe.com.
