Telling the time with audiovisual speech and non-speech: Does the brain use multiple clocks?
Alberta Ipser, Diana Paunoiu, Elliot Freeman

Date: 2012-06-19 11:00 AM – 12:30 PM
Last modified: 2012-04-24

Abstract


It has often been claimed that there is mutual dependence between the perceived synchrony of auditory and visual sources, and extent to which they perceptually integrate (‘unity assumption’: Vroomen & Keetels, 2010; Welsh & Warren, 1980). However subjective audiovisual synchrony can vary widely between subjects (Stone, 2001) and between paradigms (van Eijk, 2008). Do such individual differences in subjective synchrony correlate positively with individual differences in optimal timing for integration, as expected under the unity assumption? In separate experiments we measured the optimal audiovisual asynchrony for the McGurk illusion (McGurk & MacDonald, 1976), and the stream-bounce illusion (Sekuler, Sekuler, & Lau, 1997). We concurrently elicited either temporal order judgements (TOJ) or simultaneity judgements (SJ), in counterbalanced sessions, from which we derived the point of subjective simultaneity (PSS). For both experiments, the asynchrony for maximum illusion showed a significant positive correlation with PSS derived from SJ, following the unity assumption. But surprisingly, the analogous correlation with PSS derived from TOJ was significantly negative. The temporal mechanisms for this pairing of tasks seem neither unitary nor fully independent, but apparently antagonistic. A tentative temporal renormalisation mechanism explains these paradoxical results as follows: 1) subjective timing in our different tasks can depend on independent mechanisms subject to their own neural delays; 2) inter-modal synchronization is achieved by first discounting the mean neural delay within each modality; and 3) apparent antagonism between estimates of subjective timing emerges as the mean is attracted towards deviants in the unimodal temporal distribution.

References


McGurk, H., & MacDonald, J. (1976). Hearing lips and seeing voices. Nature, 264(5588), 746-748. Nature Publishing Group. doi:10.1038/264746a0
Sekuler, R., Sekuler, A. B., & Lau, R. (1997). Sound alters visual motion perception. Nature, 385(6614), 308. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/9002513
Stone, J. V., Hunkin, N. M., Porrill, J., Wood, R., Keeler, V., Beanland, M., Port, M., et al. (2001). When is now? Perception of simultaneity. Proceedings. Biological sciences / The Royal Society, 268(1462), 31-8. doi:10.1098/rspb.2000.1326
van Eijk, R. L. J. (2008). Audio–visual synchrony perception. Unpublished doctoral dissertation, Technische Universiteit Eindhoven, Eindhoven. Retrieved from http://alexandria.tue.nl/extra2/200810581.pdf
Vroomen, J., & Keetels, M. (2010). Perception of intersensory synchrony: a tutorial review. Attention perception psychophysics, 72(4), 871-884. Psychonomic Society Publications. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/20436185
Welch, R. B., & Warren, D. H. (1980). Immediate perceptual response to intersensory discrepancy. Psychological bulletin, 88(3), 638-67. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/7003641

Conference System by Open Conference Systems & MohSho Interactive