Recent results from human imaging and electrophysiology demonstrate that the processing of acoustic information can be influenced by stimulation of other sensory modalities already at early stages in auditory cortex. Here we scrutinize the neuronal basis of these multisensory influences at different stages along the auditory processing streams. To this end we record neuronal responses in different regions of auditory cortex and in the upper bank of the superior temporal sulcus (STS) during stimulation with naturalistic audio-visual stimuli.
In caudal primary or secondary auditory fields only few neurons revealed significant visual influences. However in regions beyond the classical auditory cortex, their fraction increased considerably, and in association regions such as the upper bank STS both bimodal neurons and neurons with significant multisensory interactions were common. In fact, our results demonstrate a spatial topographical layout of modality preferences in the STS, which might provide a neural basis of sensory integration in this region. To characterize the effect of multisensory inputs at each of these processing stages we investigate the importance of stimulus congruency on multisensory interactions. In addition, to distinguish basic response modulation from sensory integration we use methods of information theory to determine whether visual input actually enhances the information encoded by neuronal responses about the stimulus. Altogether, our results provide first insights into how the impact of multisensory influences increases along a sensory processing pathway and start to shed light on where and how the neural representation of the sensory environment benefits from multisensory inputs.