What is sensory about multi-sensory enhancement of vision by sounds?
Last modified: 2011-09-04
Abstract
Can auditory input influence the sensory processing of visual information? Many studies have reported cross-modal enhancement in visual tasks, but the nature of such gain is still unclear. Some authors argue for ‘high-order’ expectancy or attention effects, whereas others propose ‘low-order’ stimulus-driven multisensory integration. The present study applies a psychophysical analysis of reaction time distributions in order to disentangle sensory changes from other kind of high-order (not sensory-specific) effects. Observers performed a speeded simple detection task on Gabor patches of different spatial frequencies and contrasts, with and without accompanying sounds. The data were adjusted using chronometric functions in order to separate changes is sensory evidence from changes in decision or motor times. The results supported the existence of a stimulus unspecific auditory-induced enhancement in RTs across all types of visual stimuli, probably mediated by higher-order effects (e.g. reduction of temporal uncertainty). Critically, we also singled out a sensory gain that was selective to low spatial frequency stimuli, highlighting the role of the magno-cellular visual pathway in multisensory integration for fast detection. The present findings help clarify previous mixed findings in the area, and introduce a novel form to evaluate cross-modal enhancement.
References
1. Frassinetti, F., N. Bolognini, and E. Ladavas, Enhancement of visual perception by crossmodal visuo-auditory interaction. Exp Brain Res, 2002. 147(3): p. 332-43. 2. Frassinetti, F., F. Pavani, and E. Ladavas, Acoustical vision of neglected stimuli: interaction among spatially converging audiovisual inputs in neglect patients. J Cogn Neurosci, 2002. 14(1): p. 62-9. 3. Jaekl, P.M. and S. Soto-Faraco, Audiovisual contrast enhancement is articulated primarily via the M-pathway. Brain Res, 2010. 1366: p. 85-92. 4. McDonald, J.J., W.A. Teder-Salejarvi, and S.A. Hillyard, Involuntary orienting to sound improves visual perception. Nature, 2000. 407(6806): p. 906-8. 5. Odgaard, E.C., Y. Arieh, and L.E. Marks, Cross-modal enhancement of perceived brightness: sensory interaction versus response bias. Percept Psychophys, 2003. 65(1): p. 123-32. 6. Lippert, M., N.K. Logothetis, and C. Kayser, Improvement of visual contrast detection by a simultaneous sound. Brain Res, 2007. 1173: p. 102-9. 7. Giard, M.H. and F. Peronnet, Auditory-visual integration during multimodal object recognition in humans: a behavioral and electrophysiological study. J Cogn Neurosci, 1999. 11(5): p. 473-90. 8. Watkins, S., et al., Activity in human V1 follows multisensory perception. Neuroimage, 2007. 37(2): p. 572-8. 9. Watkins, S., et al., Sound alters activity in human V1 in association with illusory visual perception. Neuroimage, 2006. 31(3): p. 1247-56. 10. Kastner, S., et al., Increased activity in human visual cortex during directed attention in the absence of visual stimulation. Neuron, 1999. 22(4): p. 751-61. 11. Slotnick, S.D., W.L. Thompson, and S.M. Kosslyn, Visual mental imagery induces retinotopically organized activation of early visual areas. Cereb Cortex, 2005. 15(10): p. 1570-83. 12. Rowland, B.A. and B.E. Stein, Multisensory integration produces an initial response enhancement. Front Integr Neurosci, 2007. 1: p. 4. 13. Colonius, H. and A. Diederich, The race model inequality: interpreting a geometric measure of the amount of violation. Psychol Rev, 2006. 113(1): p. 148-54. 14. Miller, J., Divided attention: evidence for coactivation with redundant signals. Cogn Psychol, 1982. 14(2): p. 247-79. 15. Otto, T.U. and P. Masmassian, Divided Attention and Sensory Integration: The Return of the Race Model. Journal of Vision, 2010. 10(7). 16. Plainis, S. and I.J. Murray, Neurophysiological interpretation of human visual reaction times: effect of contrast, spatial frequency and luminance. Neuropsychologia, 2000. 38(12): p. 1555-64. 17. Nickerson, R.S., Intersensory facilitation of reaction time: energy summation or preparation enhancement? Psychol Rev, 1973. 80(6): p. 489-509. 18. Valls-Sole, J., et al., Reaction time and acoustic startle in normal human subjects. Neurosci Lett, 1995. 195(2): p. 97-100.