Short and sweet, or long and complex? Perceiving temporal synchrony in audiovisual events
Ragnhild Eg, Dawn Behne

Date: 2012-06-20 02:30 PM – 04:00 PM
Last modified: 2012-04-25

Abstract


Perceived synchrony varies depending on the audiovisual event. Typically, asynchrony is tolerated at greater lead- and lag-times for speech and music than for action events. The tolerance for asynchrony in speech has been attributed to the unity assumption, which proposes a bonding of auditory and visual speech cues through associations in several dimensions. However, the variations in synchrony perception for different audiovisual events may simply be related to their complexity; where speech and music fluctuate naturally, actions involve isolated events and anticipated moments of impact. The current study measured perception of synchrony for long (13 seconds) and short (1 second) variants of three types of stimuli: (1) action, represented by a game of chess, (2) music, played by a drummer, and (3) speech, presented by an anchorwoman in a newscast. The long variants allowed events to play out with their natural dynamics, whereas short variants offered controlled and predictable single actions or events, selected from the longer segments. Results show that among the long stimuli, lead asynchrony was detected sooner for speech than for chess. This contrasts both with previous research and our own predictions, although it may be related to characteristics of the selected chess scene. Interestingly, tolerance to asynchrony was generally greater for short, than for long, stimuli, especially for speech. These findings suggest that the dynamics of complex events cannot account for previously observed differences in synchrony perception between speech and action events.

References


Arrighi, R., Alais, D., & Burr, D. (2006). Perceptual synchrony of audiovisual streams for natural and artificial motion sequences. Journal of Vision, 6(3), 260-268.

Dixon, N. F., & Spitz, L. (1980). The detection of auditory visual desynchrony. Perception, 9, 719-721.

Fujisaki, W., & Nishida, S. (2005). Temporal frequency characteristics of synchrony-asynchrony discrimination of audio-visual signals. Experimental Brain Research, 166(3-4), 455-464.

Liberman, A. M. (1982). On finding that speech is special. American Psychologist, 37(2), 148-167.

Vatakis, A., Ghazanfar, A. A., & Spence, C. (2008). Facilitation of multisensory integration by the “unity effect” reveals that speech is special. Journal of Vision, 8(9), 1-11.

Vatakis, A., & Spence, C. (2006). Audiovisual synchrony perception for music, speech, and object actions. Brain Research, 1111(1), 134-142.

Vatakis, A., & Spence, C. (2007). Crossmodal binding: Evaluating the “unity assumption” using audiovisual speech stimuli. Perception & Psychophysics, 69(5), 744-756.

Conference System by Open Conference Systems & MohSho Interactive