Multisensory attentional-spread activity reveals tighter audiovisual-synchrony constraints for the spatial shifts of ventriloquism than for the temporal linking of auditory and visual events

Sarah E. Donohue, Maria A. Pavlova, Kenneth C. Roberts, Tineke Grent-'t-Jong, Marty G. Woldorff
Poster
Time: 2009-06-30  09:00 AM – 10:30 AM
Last modified: 2009-06-04

Abstract


A fundamental task in daily life is the accurate perception and integration of information from multiple modalities into discrete objects. This can be done in a robust and reliable manner by using cues from space and time, with stimuli more proximal in space and/or time more likely to be arising from the same object and thus needing to be linked together and integrated. On the other hand, auditory and visual stimuli can be temporally linked together when they occur within a relatively wide temporal window of integration, and during the ventriloquism illusion the spatial location of an auditory stimulus can be perceptually drawn toward that of a simultaneously occurring visual stimulus. Attention appears to play a key role in these processes by aiding in the grouping of sensory information, whether it be within a single modality or across modalities. Moreover, it has been shown that attention to stimuli in one modality (vision) can spread to irrelevant but synchronous stimuli in another modality (audition), even when they arise from different locations, an effect reflected by a late (>200 ms), long-lasting (200-700 ms) frontal ERP negativity and an enhanced level of auditory-cortex fMRI activity (Busse et al., 2005).

Using EEG in 18 healthy participants, we investigated such attentional spread when the irrelevant auditory event was either physically simultaneous with the visual, delayed by 100 ms (inside the temporal window of integration), or delayed by 300 ms (outside the temporal window of integration), with all cases having spatial separation between the auditory and visual stimuli. When the irrelevant auditory stimulus was synchronous with an attended versus an unattended visual event, the late frontal negativity was enhanced, replicating Busse et al. (2005). When the auditory stimulus was delayed by 100 ms, the main sustained part of late frontal negative wave was slightly attenuated relative to the simultaneous condition, and shifted in time by 100 ms. When the auditory stimulus was delayed by 300 ms, the late negative wave was highly attenuated and shifted by 300 ms. Further, when the auditory and visual stimuli occurred simultaneously, the attentional spread ERP activity began with an additional fronto-central negative-wave component at 200 ms, an activation that was not seen in the 100-ms or 300-ms delay conditions. This initial negative component occurred contralateral to the side of the attended visual stimulus, likely reflecting the neural underpinnings of the perceptual shift in the spatial location of the sound toward that visual stimulus. This effect was similar in timing and distribution to that observed by Bonath et al., 2007; here, however, it was only found as a function of multisensory attention (i.e., from the difference wave between conditions when the auditory stimulus occurred with an attended versus an unattended visual event). In contrast, the more sustained, noncontralateral, negative-wave activity elicited for both the simultaneous and the 100-ms delay conditions appears to be reflective of the multisensory temporal linking than can take place for visual and auditory events that occur across a broader time window. These results provide novel insight into the temporal and spatial linking of the unisensory components of multisensory stimuli, and the way in which attention interacts with these perception-related stimulus-linking processes.

Conference System by Open Conference Systems & MohSho Interactive Multimedia