Skip page navigation

Skip page navigation

Mismatch in Human Realism of Face and Voice of Human Looking Animated Characters, Robots Make Us Feel Creepy

Karl MacDorman
Karl MacDorman


October 31, 2011

Contact Information:

View Related Releases:

Share This:

  • Share

It sounds like a location in a Stephen King novel, but for producers of animated characters or robots that look imperfectly human it is all too real: the uncanny valley.

Uncanny valley is a term coined more than 40 years ago by Japanese roboticist Masahiro Mori to describe the eerie feeling of viewing an animated character or robot that looks imperfectly human.

In what seems like true Stephen King fashion, Mori proposed the more human the character looks the more comfortable people feel interacting with it until a point is reached at which subtle nonhuman flaws cause the character to seem eerie, like an animated corpse.

A new study examines whether a mismatch between the human realism of a character’s face and voice causes it to be evaluated as eerie. The study concludes this mismatch causes the uncanny valley effect as well.

In a recent paper on the study (, Karl MacDorman, Ph.D., Associate Professor, Human-Computer Interaction, and Director, Android Science Center at the IU School of Informatics at Indiana University-Purdue University Indianapolis, describes an experiment in which 48 university undergraduate students watched four videos of a character reciting neutral phrases. Each video corresponded to either the matched (robot figure–synthetized voice, human figure–human voice) or mismatched condition (robot figure–human voice, human figure–synthetized voice). The results show a human voice heightened the eeriness of the robot, while a synthetized voice heightened the eeriness of the human.

MacDorman’s previous studies have shown how relations among a character’s visual elements – such as cartoonish eyes and human-looking skin – cause the uncanny valley effect.
Mismatches among visual elements or between the human realism of a character’s face and voice have significant implications beyond feelings of eeriness, MacDorman said. They can spell the difference between the success and failure of a movie, videogame, or website that uses animated human characters.

According to MacDorman, it is easy for viewers to identify with an abstract character like Mickey Mouse, but ironically not with a character that looks completely human except for a nonhuman flaw.

“Mismatches suppress the empathy of viewers,” MacDorman said. “They can no longer relate to the character vicariously.”

The reasons for that, MacDorman believes, may have to do with a tension between “what you consciously know about the character and what the unconscious processing of the brain is telling you. Different processes in the brain may be forming different conclusions about whether the character is human or nonhuman, alive or dead.”

As challenging as hurdling the uncanny valley is there have been examples of where it has been successfully bridged, even if only in a limited fashion.

MacDorman points to two films, The Matrix and The Curious Case of Benjamin Button as examples of overcoming the uncanny valley. They succeeded, he said, because “animated characters were used in a limited way, the animation was done very well and other aspects of the film were really authentic," he said.

"It also made sense to use computer animation in these films because their plots demanded that human characters do things that aren’t humanly possible—and the roles were too serious to be played by a cartoon character,” MacDorman continued.

An example of a film that wound up in the uncanny valley is Mars Needs Mom, due to issues related to the mismatch between the way the characters looked and moved, stemming from its motion capture technology.

There are increasing uses of animated characters that greet visitors to websites and offer advice or information. While they look very human, with one character on a financial website produced in a manner that makes the character look very distinguished with grey hair, not all of what they may have to say to the website viewer can be pre-recorded. When statements are needed that can’t be recorded, a synthetized voice is used.

When that happens, it runs afoul of what MacDorman’s study recommends: the human realism of a character’s visual elements and voice should match. Otherwise, you are bound for the uncanny valley.