Does optimal integration of auditory and visual cues occur in a complex temporal task?
Scott Love, James M. Hillis, Frank E Pollick
Poster
Last modified: 2008-05-13
Abstract
When multiple sources of sensory information about a single environmental property are available, combining them can form improved estimates of that property. For simple physical dimensions such as object size (Ernst & Banks, 2002) and location (Alais & Burr, 2004), studies using a standard paradigm, show that humans integrate different sensory sources in a statistically optimal fashion. It is unclear whether this paradigm can be used to understand integration of auditory and visual cues in the temporal domain (Roach, Heron & McGraw, 2006). We investigated this question whilst using a more complex stimulus: drumming point light displays. Stimuli were created from 3D motion capture data of a drummer performing swing groove drumming, which was converted into visual point light displays. Sounds were obtained by simulation of 25 modes of a circular membrane. Parameters for the sound model were the physical parameters of the membrane and the time and impact velocity of a strike. There were three main conditions in the experiment: audio-alone, vision-alone and audio-visual combined. Auditory noise was added to the audio-alone and audio-visual conditions and discrepancies between the cues occurred in the audio-visual. For each of these conditions, we measured tempo discrimination performance in a 2IFC task.