Audiovisual speech perception: acoustic and visual phonetic features contributing to the McGurk effect
Kaisa Tiippana, Martti Vainio, Mikko Tiainen

Last modified: 2011-09-02

Abstract


In the best-known example of the McGurk effect, an auditory consonant /b/ that is presented with a face articulating /g/ is heard as a fusion /d/. However, sometimes this kind of stimulus is heard as /g/, i.e. a visual-dominant percept. We explored the stimulus features giving rise to these percepts by using two different stimuli at various levels of acoustic noise. The stimulus auditory /apa/ presented with visual /aka/ was most often heard as /aka/ even without noise, and the proportion of visual-dominant percepts increased with noise level. The stimulus auditory/epe/ presented with visual/eke/ was heard mostly as a fusion /ete/, except at the highest noise level where also /eke/ was heard. The differences in the quality of the McGurk effect were accounted for by the features of the unisensory stimuli. A phonetic analysis showed that the auditory and visual stimulus features were close to those of /t/ in the /e/-context, but not in the /a/-context stimuli. Thus, the type of the McGurk effect - fusion or visual dominance - depended on the quality of the constituent stimulus components, obeying a modality-precision rule: the more reliable the unisensory stimulus, the greater its contribution to the final percept.

Conference System by Open Conference Systems & MohSho Interactive