< Back to front page Text size +
MIT

People can hear with their skin, study shows.

Posted by Gideon Gil  November 25, 2009 02:24 PM
  • Facebook
  • E-mail
  • E-mail this article

    Invalid E-mail address
    Invalid E-mail address

    Sending your article

    Your article has been sent.

E-mail this article

Invalid email address
Invalid email address

Sending your article

Your article has been sent.

By Carolyn Y. Johnson, Globe Staff

Listening is more than a matter of being "all ears." People can also hear with their skin, according to new research that deepens our understanding of the senses, showing they can work together but also override one another.

Strange though it seems, scientists are finding that multiple senses contribute to the simplest perceptions. People can see with their ears, hear with their eyes, or hear with a touch.

In the work published today in the journal Nature, researchers found they could influence what people hear by delivering puffs of air to the back of a hand or their neck. By demonstrating that the perception of speech is affected by touch, the experiment raises the possibility that one sense could be used as a substitute for another, creating new ways for deaf people to hear. In fact, researchers at MIT are already using this basic idea to develop technology that could one day assist people with hearing impairment.

"This study is part of us ... reconsidering how humans perceive the world, how humans interact with the world," said co-author Bryan Gick, a phonetician at the University of British Columbia and senior scientist at Haskins Laboratories, a speech research think tank in New Haven, Conn.

For years, scientists have known that watching another person speak can affect what we hear. In a well-known phenomenon called the McGurk effect, a person who listens to audio of someone saying "ba ba ba", while watching another person's lips forming the words "ga ga ga," hears something in-between: "da da da." (Watch this video for a demonstration.)

Now, Gick is exploring whether touch also affects hearing. In the experiment, subjects heard the sounds "pa" or "ba" and "ta" or "da." Sometimes, they received a puff of air on the back of their hand or neck when the words had an aspirated sound -- a sound like "pa" or "ta" that requires the speaker to expel a puff of air. (Hold your hand to your mouth and say "pa" and compare it to "ba" to feel the difference). Other times, they got the reverse: a puff of air when they heard "ba" or "da" -- non-aspirated sounds.

The researchers found that when the puff of air was paired with the aspirated word, people got better at identifying the sound. When the puff of air was paired with "ba" or "da," accuracy declined.

"This is a very intriguing finding, raising lots of theoretical possibilities and future studies," said Shinsuke Shimojo, head of a psychophysics laboratory at California Institute of Technology. He said it was interesting that the puff seemed to be an implicit signal for one sound over another, and worked at both spots on the body. The people were not told or taught that the puff was meant to signal a certain sound, and the researchers tried to choose spots on the body that likely did not feel aspirated puffs when people were speaking.

While it is not yet clear what is happening in the brains of people who "hear" puffs of air, the experiment supports the idea that people integrate information from touch with sounds they hear. In a study published in the journal NeuroImage three years ago, Finnish researchers used brain imaging to study 13 subjects and found that touch activated the auditory cortex, a part of the brain that is involved in hearing.

Charlotte Reed, a senior research scientist at the Massachusetts Institute of Technology, specializes in researching the ways in which touch can be used to interpret speech, by studying deaf-blind people who learn the Tadoma method -- a way of learning to talk and hear by placing a hand on the neck and mouth of a speaker.

"We know the auditory and tactile senses interact," Reed said. She pointed out that the new study certainly shows that touch can give a conflicting signal, making someone hear a "ba" as a "pa," but also shows that a person's accuracy in understanding a sound can be enhanced. Using touch to enhance hearing is the basic idea behind her research, which uses a machine called the "tactuator" that involves three vibrating and moving prongs -- one for the thumb, forefinger, and middle finger to rest on -- to turn speech into something people feel.

The idea is to create an aid for lipreading, and eventually to use the information gleaned from the bulky, three-pronged device to create software that could one day be used to turn a simple device like a cellphone into a prosthetic for deaf people. The microphone on a cellphone could translate a speaker's voice into tactile signals that could help a person understand someone as they were lip-reading.

Research on how the senses intersect in productive and conflicting ways goes beyond touch and speech.

Earlier this year, other researchers published research showing that people hear through their face. Stretching a person's facial muscles into shapes normally associated with speech could make them hear words differently. For example, when the researchers played an ambiguous word, they found that people heard the word as "head" when their facial skin was stretched upwards, and heard "had" when their facial skin was stretched downwards.

Other researchers have described the parchment-skin illusion, in which people rub their palms together next to a microphone and hear the sound through earphones. When scientists enhanced the rubbing noises, people reported their hands felt dry, like parchment paper -- an example of feeling with their ears. In another study, researchers found that by simply tweaking the sound of people crunching on a potato chip, listeners rated the chip as more crispy.

CalTech's Shimojo studies vision and has demonstrated that people shown a flash of light accompanied by a beep will report seeing two flashes if they then hear two beeps but see only one flash.

Gick says his research has potential for helping people hear using touch, but for now he hopes to use brain imaging to reveal what is happening when people "hear" an air puff.

Taken together, such work is more than just a collection of cross-sensory illusions: it suggests something fundamental about how our brains work.

"It's starting to look more and more like we're perceiving machines," Gick said. "Are we physically constructed to take in information, irrespective of where it's coming from or what form it's in?"

  • Facebook
  • E-mail
  • E-mail this article

    Invalid E-mail address
    Invalid E-mail address

    Sending your article

    Your article has been sent.

About white coat notes

White Coat Notes covers the latest from the health care industry, hospitals, doctors offices, labs, insurers, and the corridors of government. Chelsea Conaboy previously covered health care for The Philadelphia Inquirer. Write her at cconaboy@boston.com. Follow her on Twitter: @cconaboy.
health answers

Long-term health consequences to being born prematurely? It's estimated that each year nearly 500,000 babies in the United States are born prematurely, or before 37 weeks of pregnancy. Submit question | More answers

Health&Wellness video

archives