From visual symbols to sound representations: Event-related potentials and gamma-band responses

Erich Schröger, Andreas Widmann, Thomas Gruber
Talk
Last modified: 2008-05-13

Abstract


We studied audio-visual integration with event-related potentials (ERPs) and gamma-band responses (GBRs). Human subjects performed symbol-to-sound-matching paradigm in which score-like visual stimuli had to be mapped to corresponding sound patterns. The sounds could be either congruent or occasionally incongruent with the corresponding symbol. In response to congruent sounds, a power increase of phase-locked (evoked) GBR in the 40-Hz band was observed peaking 42-ms post-stimulus onset. This suggests that the comparison process between an expected sound and the current sensory input is implemented at early levels of auditory processing. Subsequently, expected congruent sounds elicited a broadband power increase of non-phase-locked (induced) GBR peaking 152-ms post-stimulus onset, which might reflect the formation of a unitary event representation including both visual and auditory aspects of the stimulation. GBRs were not present for unexpected incongruent sounds. However, incongruent sounds elicited an ERP component starting at about 100 ms relative to their onset. It had a bilateral frontal distribution and a polarity inversion at the mastoids pointing at sources in auditory areas. Results can be explained by a model postulating the anticipatory activation of cortical auditory representations and the match of experience against this expectation. GBRs are sensitive to a match of the forward model, ERPs to a mismatch.

Conference System by Open Conference Systems & MohSho Interactive Multimedia