Visual-to-auditory substitution learning: Behavioral findings and neural correlates

Jung-Kyong Kim, Montreal Neurological Institute, McGill University

Abstract
We investigated visual-to-auditory substitution learning in sighted subjects using the conversion algorithm developed by Meijer (1992). Over the course of approximately three weeks, eight sighted subjects participated in nine ~1.75-hour training sessions during which they learned the relationship of various visual images with their corresponding converted sounds. The visual images included abstract figures, pictures of real-life objects, and pictures of scenes. Subjects were evaluated with forced-choice recognition tasks at the end of every three training sessions using both the items encountered during training and new items designed to test for generalization. Significant improvement was observed over time in recognizing the corresponding converted sounds for novel visual images, with the biggest improvement found in identification of the abstract figures. Drawing tasks also demonstrated subjects’ abilities to extract overall shapes of the visual images from the corresponding converted sounds. Based on these findings, we are conducting an fMRI study using the same training protocol for a period of five consecutive days. We expect that the behavioral improvement in visual judgments of the converted sounds will be accompanied by changes in occipital cortex activity as a result of the structured training. Such findings would indicate that cross-modal cortical interactions underlie visual-to-auditory substitution learning.

Not available

Back to Abstract