ERP evidence for early cross-modal integration in visual selection

Jan Theeuwes, Erik van der Burg, Chris Olivers, Durk Talsma, Adelbert Bronkhorst
Talk
Time: 2009-07-02  11:50 AM – 12:10 PM
Last modified: 2009-06-04

Abstract


Recently we have demonstrated that synchronized auditory signals can greatly increase the saliency of visual target events in cluttered, continuously changing displays [Van der Burg, Olivers, Bronkhorst, and Theeuwes. (2008) Journal of Experimental Psychology: Human Perception and Performance. The sound makes the visual target pop out. This “pip and pop� effect occurs even though the sound carries no information on the location or identity of the target, as long as it is synchronized with the visual event. Here we report evidence from EEG data that the pip and pop effect results from audio-visual integration occurring at the early (preattentive) stages of processing. The audio-visual integration boosts the salience of visual signal such that it captures attention. Participants performed a visual search task with displays consisting of a multitude of oblique bars that continuously flipped between different orientations. The target was a bar that changed to horizontal or vertical, and the task was to indicate its orientation with an unspeeded response. Accuracy data showed the occurrence of the pip and pop effect: observers were better able to detect the target when it was accompanied by a sound, compared to when no sound was present, or when the sound was synchronized with a distractor instead. EEG analysis revealed an early modulation of the event-related potential (ERP) around 50 ms from target onset, when a tone was synchronized with the target, compared to the summed
activity of those conditions in which only the auditory or only the visual signal was present. Around 250 ms, a lateralized ERP component (the N2pc) to the target was observed reflecting the capture of spatial attention to the location of the target. This component was followed by an increased P3 reflecting target identification. We conclude that audio-visual integration boosts the salience of the visual event causing it to gain priority over competing visual events.

Conference System by Open Conference Systems & MohSho Interactive Multimedia