Crossmodal perceptual learning and sensory substitution
Michael J Proulx

Last modified: 2011-08-22

Abstract


A sensory substitution device for blind persons aims to provide the missing visual input by converting images into a form that another modality can perceive, such as sound. Here I will discuss the perceptual learning and attentional mechanisms necessary for interpreting sounds produced by a device (The vOICe) in a visuospatial manner. Although some aspects of the conversion, such as relating vertical location to pitch, rely on natural crossmodal mappings, the extensive training required suggests that synthetic mappings are required to generalize perceptual learning to new objects and environments, and ultimately to experience visual qualia. Here I will discuss the effects of the conversion and training on perception and attention that demonstrate the synthetic nature of learning the crossmodal mapping. Sensorimotor experience may be required to facilitate learning, develop expertise, and to develop a form of synthetic synaesthesia.

References


Chebat, D. R., Rainville, C., Kupers, R., & Ptito, M. (2007). Tactile-‘visual’ acuity of the tongue in early blind individuals. Neuroreport, 18, 1901-1904.

Meijer, P. B. L. (1992). An experimental system for auditory image representations. IEEE Transactions on Biomedical Engineering, 39, 112-121.

Proulx, M. J. (2010). Synthetic synaesthesia and sensory substitution. Consciousness and Cognition, 19, 501-503.

Proulx, M. J. & Stoerig, P. (2006). Seeing sounds and tingling tongues: Qualia in synaesthesia and sensory substitution. Anthropology & Philosophy, 7, 135-151.

Proulx MJ, Stoerig P, Ludowig E, Knoll I.Seeing 'where' through the ears: effects of learning-by-doing and long-term
sensory deprivation on localization based on image-to-sound substitution. PLoS One. 2008 Mar 26;3(3):e1840.

Ward J, Meijer P. Visual experiences in the blind induced by an auditory sensory substitution device.Conscious Cogn.
2010 Mar;19(1):492-500.

Conference System by Open Conference Systems & MohSho Interactive