Mouth reversal extinguishes Mismatch Negativity induced by the McGurk Illusion
Last modified: 2013-05-05
Abstract
The sight of articulatory mouth movements (visual speech) influences auditory speech perception. This is demonstrated by the McGurk illusion in which incongruent visual speech alters the auditory phonetic percept.
In behavioral studies, reversal of the vertical mouth direction has been reported to greatly reduce the McGurk Illusion (Rosenblum et al., 2000). Such findings support the idea that audiovisual integration in speech to some extent relies on information regarding facial configuration.
Here we ask whether this behavioral effect is reflected in a difference in neural activity in the auditory cortex. Mismatch Negativity (MMN) is a component in the auditory Event-Related Potential (ERP) that is elicited by a change in the auditory percept. It has been shown that the McGurk Illusion can induce a MMN.
We conducted an experiment in which the MMN could be elicited by the McGurk Illusion induced by visual speech with either upright (unaltered) or vertically reversed mouth area.
In a preliminary analysis, we found a Mismatch Negativity component induced by the McGurk Illusion for 6 of 17 participants at electrode Cz when the mouth area was upright. In comparison, these participants produced no Mismatch Negativity when the mouth was reversed. These findings mirrored behavioral findings in the same subjects of a strong McGurk response for normal audiovisual speech, which was greatly reduced for stimuli with reversed mouth area.
In behavioral studies, reversal of the vertical mouth direction has been reported to greatly reduce the McGurk Illusion (Rosenblum et al., 2000). Such findings support the idea that audiovisual integration in speech to some extent relies on information regarding facial configuration.
Here we ask whether this behavioral effect is reflected in a difference in neural activity in the auditory cortex. Mismatch Negativity (MMN) is a component in the auditory Event-Related Potential (ERP) that is elicited by a change in the auditory percept. It has been shown that the McGurk Illusion can induce a MMN.
We conducted an experiment in which the MMN could be elicited by the McGurk Illusion induced by visual speech with either upright (unaltered) or vertically reversed mouth area.
In a preliminary analysis, we found a Mismatch Negativity component induced by the McGurk Illusion for 6 of 17 participants at electrode Cz when the mouth area was upright. In comparison, these participants produced no Mismatch Negativity when the mouth was reversed. These findings mirrored behavioral findings in the same subjects of a strong McGurk response for normal audiovisual speech, which was greatly reduced for stimuli with reversed mouth area.
Keywords
Audiovisual integration; Speech Perception; McGurk Illusion; Event-Related Potentials; Mismatch Negativity
References
Näätänen, R., Gaillard, A.W.K., and Mäntysalo, S. (1978). Early selective-attention effect on evoked potential reinterpreted. Acta Psychologica 42, 313–329.
Rosenblum, L.D., Yakel, D.A., and Green, K.P. (2000). Face and mouth inversion effects on visual and audiovisual speech perception. Journal of Experimental Psychology: Human Perception and Performance 26, 806–819.
Rosenblum, L.D., Yakel, D.A., and Green, K.P. (2000). Face and mouth inversion effects on visual and audiovisual speech perception. Journal of Experimental Psychology: Human Perception and Performance 26, 806–819.