The Voice Your Eyes Help Invent
A mismatched mouth movement can change the syllable a listener reports hearing. A classic experiment reveals how sight joins sound in building a voice.
Two Cues, One Voice
A mouth appears to say one syllable while the soundtrack plays another. The resulting sound can seem unlike either cue on its own. The ears receive sound; the eyes help identify it.
Harry McGurk and John MacDonald described this audiovisual speech illusion in their 1976 Nature paper, Hearing lips and seeing voices. A mismatched view of a speaker's lips can change the syllable a listener reports hearing. This is an experimental effect, not evidence that the recording has changed.
The Construct Behind Conversation
Ordinary speech arrives with more than a waveform. We also watch mouths move. The McGurk effect makes that integration noticeable by making the seen and heard components disagree.
- The visual cue is mouth movement, not another sound.
- The reported percept can reflect a combination of sight and hearing.
- The original paper does not show that every listener hears the same thing every time.
A Seam in Perception
JitterSnap's interpretation: the voice we experience is partly constructed from the scene around it. Turn away and the visual cue disappears. That does not change the sound wave, rewrite memory or reveal a simulation. It exposes one seam in perception under controlled conditions.
The soundtrack stays put. The syllable you hear can move.