Crossmodal compensation during face-voice integration in cochlear implanted deaf patients.
Laure Chambaudie, Pascal Belin, Thibault Briere, Chris James, olivier Deguine, Pascal Barone

Last modified: 2011-08-24

Abstract


Any dysfunction in the capacity of voice or face recognition can negatively impact on the social communication of a person, this is particularly the case in profoundly deaf patients. A cochlear implant (CI) allows deaf patients to understand speech but because of the limitations of the processor, patients present strong difficulties in voice recognition. Here we investigated the possibility that the visual system can exert in CI users a strong influence on the multimodal perception of voice attributes. We used a speaker discrimination task between sounds taken from a voice continuum obtained by morphing between a male and a female voice. Proficient CI patients (n=10) were tested under auditory-only or audiovisual conditions in which a female or male face was simultaneously presented. Their performance was compared to those of normal hearing subjects (NHS n=50) tested with a vocoded voice stimuli that simulate the processing of an implant. A visual impact index computed from the A and AV psychometric functions revealed that CI users are significantly influenced by visual cues. This is expressed by a shift in categorization of the voice toward the gender carried by the face in incongruent AV conditions. No such visual effect was observed in NHS tested with the vocoder in spite of a deficit in the A-only categorization. Thus, in case of ambiguity in the stimuli and uncertainty in the auditory signal, CI users perceptual decisions are based mainly on vision their most reliable sensory channel. These results, coupled to our brain imaging study showing in CI patient a functional colonization of the voice sensitive areas by visual speechreading, suggest a crossmodal reorganization of the mechanisms of face-voice integration after a prolonged period of deafness.

Conference System by Open Conference Systems & MohSho Interactive