Bayesian inference underlies human auditory-visual perception: a unifying ac-count of sensory integration-segregation

Ladan Shams, UCLA Department of Psychology

Abstract
Temporally coincident signals in the different sensory modalities do not always originate from the same source, and thus, should not—and do not—always get integrated. However, previous models of cross-modal interactions have exclusively focused on conditions in which the signals of the different modalities do get fused, and are unable to account for conditions in which the signals do not get integrated. We developed an ideal observer model which uses Bayesian inference to make inference about the causes of the various sensory signals. We tested the model in a paradigm in which subjects were asked to report the number of brief flashes and beeps. The human observers’ auditory-visual perception was surprisingly consistent with the ideal observer, indicating that the rule used by the nervous system for when and how to combine auditory and visual signals is statistically optimal. These results provide the first unifying account for the entire spectrum of cue combination, ranging from no integration, to partial interactions, to complete fusion. Our findings also show that the sound-induced flash illusion (in which a single flash is perceived as multiple when accompanied by multiple sounds) is an epiphenomenon of this general, statistically optimal strategy, as opposed to a processing error.

Not available

Back to Abstract