Audiovisual temporal integration: a freezing effect from rhythms with continuously shifting phases

Manuel Vidal

Last modified: 2013-05-05

Abstract


We constantly experience multimodal events for which signals are transmitted and processed at different speeds. Deciding whether these signals are synchronous or not requires temporal flexibility in the sensory integration. Several mechanisms have been described, from recalibration following exposure to constant lags (1,2) to temporal ventriloquism in which the perceived visual onset is shifted to that of the nearest auditory onset (3). While testing recalibration used constant phase-shifts, we designed stimuli in which the auditory and visual rhythms have slightly different frequencies resulting in lags increasing or decreasing constantly. In each trial, participants viewed 7 flashes and heard 5 tone pips presented roughly within the first 5 flashes period. The task was to report whether the rhythm of the last two flashes was faster or slower than the first five. We manipulated the relative rhythm (auditory faster or slower) and final phase (5th flash at 0° or 180°). The temporal frequencies where chosen so that initial and final lags roughly went from opposite to perfect synchronization or vice-versa. We found that the perceived visual pace accelerated when captured by a faster auditory rhythm or released from a slower auditory rhythm, and decelerated when captured by a slower auditory rhythm or released from a faster auditory rhythm. Consistent with temporal ventriloquism, we evidenced a temporal freezing effect in which the perceived visual pace successively speeds up and slows down according to the ongoing audiovisual phase shift.

Keywords


temporal window of integration; audiovisual; ventriloquist effect

References


1. Fujisaki, W., Shimojo, S., Kashino, M. & Nishida, S. Recalibration of audiovisual simultaneity. Nat Neurosci 7, 773–778 (2004)
2. Vroomen, J., Keetels, M., De Gelder, B. & Bertelson, P. Recalibration of temporal order perception by exposure to audio-visual asynchrony. Brain Res Cogn Brain Res 22, 32–35 (2004)
3. Vroomen, J., De Gelder, B. & Vroomen, J. Temporal ventriloquism: sound modulates the flash-lag effect. J Exp Psychol Hum Percept Perform 30, 513–518 (2004)

Conference System by Open Conference Systems & MohSho Interactive