People exhibit remarkable sensitivity to point-light displays of human movement. The Superior Temporal Sulcus is strongly implicated in the visual perception of human action. Additionally, STS has been implicated in the integration of signals across modalities, potentially resulting in the formation of multimodal action representations. On these bases, we hypothesized that the addition of auditory cues corresponding to actions depicted in point-light displays would improve visual sensitivity to those actions.

To test this hypothesis, a psychophysical study was conducted wherein participants detected the presence of masked point-light walkers under unmoral or audiovisual conditions. Participants in audiovisual conditions heard either tones or actual footfalls coincident with the walkers’ footsteps. Results revealed improvement in detection sensitivity when visual displays were paired with veridical auditory cues (footfalls), but not when paired with simple tones, suggesting that coincident cues are not sufficient to improve visual sensitivity to actions with which they do not naturally co-occur.

To investigate the role of temporal coincidence in the sensitivity benefit conferred for coincident/veridical footstep pairings, a second study was conducted wherein participants heard tones or footfalls that were either coincident with the walkers’ footsteps or heard at random times within each trial. Two-way ANOVA revealed a main effect of sound, but no main effect of timing and no interaction. Observed increases in visual sensitivity were driven by the semantic relationship, and not by the temporal correspondence, between modality-specific cues. Under some conditions, integration of multimodal cues to human action may occur irrespective of temporal asynchronicity.