The ventriloquist effect in time is consistent with optimal combination across senses
Single Paper Presentation

David Burr
Department of Psychology, University of Florence

Martin Banks
Department of Optometry, University of California, Berkeley

Concetta Morrone
Facolta' di Psicologia, Universita' Vita-Salute "San Raffaele", Milan, Italy

     Abstract ID Number: 37
     Full text: Not available
     Last modified: March 12, 2006
     Presentation date: 06/18/2006 5:30 PM in Hamilton Building, McNeil Theatre
     (View Schedule)

Abstract
The “ventriloquist effect” (mislocalization of sound toward a visual stimulus) is consistent with optimal integration of visual and auditory signals (Alais & Burr, Curr. Biol., 2004). Here we report that “temporal ventriloquism”, the tendency for a sequence of sounds to “capture” visual flashes in time (Shams & Shimojo, Nature, 2000), is also consistent with optimal integration. Subjects performed a temporal bisection task, reporting whether the second (probe) stimulus in a 3-stimulus (800 ms) sequence was closer in time to the first or third. In single-cue sessions, the three stimuli were all either visual flashes or auditory tones. In the double-cue sessions, all stimuli comprised both flash and tone, presented simultaneously for the probe stimulus, but at consistently different times for the first and third stimuli. The perceived point of bisection in the double-cue condition was determined more by tone than flash timing, but both cues influenced the bisection. The results were well predicted from optimal combination of the visual and auditory cues with relative weights derived from the single-cue conditions. Importantly, bisection thresholds in the bimodal condition were significantly better than in either single-cue condition, strong evidence for inter-modal combination. Further discrimination experiments suggested that combination was not mandatory (Hillis et al., Science, 2002), but that subjects retain access to single-cue information as well as the combined information, calling into question the concept of mandatory temporal binding.