Role of semantic vs spatial congruency in a bimodal go/no-go task
Poster Presentation

Clara Suied
CNRS-UPMC UMR 7593, La Salpetrière Hospital, Paris, France

Nicolas Bonneel
REVES/INRIA, Sophia-Antipolis, France

Isabelle Viaud-Delmon
CNRS-UPMC UMR 7593, La Salpetrière Hospital, Paris and IRCAM, Paris, France

     Abstract ID Number: 128
     Full text: Not available
     Last modified: June 15, 2007
     Presentation date: 07/07/2007 10:00 AM in Quad Maclauren Hall
     (View Schedule)

Abstract
In virtual environments, both spatial and semantic characteristics might be at stake for optimal bimodal integration. We conducted an experiment to study how these two factors are intermingled. We assessed the role of spatial congruence between visual and auditory information in object recognition.
We used a go/no-go paradigm with two meaningful objects, either visual and/or auditory. One was the target (a phone) and the other a distractor (a train). These objects were displayed in two spatial conditions, with or without spatial disparity. The visual stimulus was embedded in an “ecological” VE representing a room.
We used a passive stereoscopic screen and Wave Field Synthesis (WFS) audio rendering, which allowed accurate reproduction of the cues of spatial sound localization.
The results demonstrate a bimodal facilitation effect in the semantically congruent case compared to both unimodal auditory and visual conditions. Moreover, reaction times in the case of semantically congruent cross-modal stimuli were faster than in the case of semantically incongruent stimuli. Surprisingly, spatial congruency had no effect on subjects' responses.
Thus, this experiment suggests a dominance of semantic congruence over spatial congruence. This finding constitutes experimental evidence of the importance of semantic content on multisensory integration.