Audiovisual fusion or just an illusion?
Azra Nahid Ali
Poster
Last modified: 2008-05-09
Abstract
McGurk fusion – ba(audio)/ga(visual) eliciting /da/ (fusion), shows the bimodality of speech perception, and has been investigated extensively for thirty years embedded in various language contexts. However, most researchers have neglected the velar fusion: ba/da eliciting /ga/ fusion and pa/ta eliciting /ka/ fusion, which resulted in fusion rates of 27% and 50% respectively (MacDonald and McGurk, 1978:255). Schwartz (2001), experimenting with only voiceless consonants, claimed that these latter types of fusion are laboratory curiosities which do not occur embedded in French syllables. In this paper we show that such velar perceptions are not just a product of isolated nonsense syllables, but are robust percepts formed by integration of audio and visual channels with high fusion rates, even when the incongruent inputs are embedded in various languages. Our evidence, derived from nonsense syllables and from real words in English, German and Arabic, covers both voiceless and voiced consonants. We further discuss how speech perception theories can be modified to model these velar fusion phenomena. For future studies we propose brain imaging methods, which have the potential to locate fusion events in the neural substrate of subjects from a wide range of language cultures and establish a degree of language-universality.