The Development of Multisensory Integration
David J. Lewkowicz, Mark Wallace, Robert Lickliter, Lorraine Bahrick, Asif A. Ghazanfar, Bruce D. McCandliss
Time: 2009-07-01 10:30 AM – 12:30 PM
Last modified: 2009-06-04
Abstract
To achieve a coherent perceptual synthesis, developing organisms must be able to perceive the intersensory relations inherent in the sensory attributes that specify their multisensory world. This symposium explores the comparative aspects of the development of multisensory integration by considering this process in three different species: human infants, monkeys, and cats and focuses on the development of multisensory integration at the behavioral and neural levels. Lickliter and Bahrick will discuss the importance of intersensory redundancy in directing selective attention in human infants and will delineate a model that predicts under what conditions perceivers selectively attend to redundantly specified vs. nonredundantly specified stimulus attributes. Lewkowicz will discuss recent findings from his laboratory showing for the first time that human infants begin life with a broadly tuned perceptual system that is sensitive to native as well as nonnative multisensory inputs and that as development progresses this sensitivity narrows as a function of experience with native multisensory inputs. Ghazanfar will consider the evolution of multisensory systems with an eye towards determining whether comparative studies of their ontogeny can provide insights into behavioral and neural homologies and will show that developmental timing and experience differentially affect the formation of multisensory circuits across different species. Wallace will discuss his findings on the development of multisensory circuits at the cortical and subcortical levels and will show that these circuits are not present at birth, mature over time, and are dramatically shaped by early sensory experience. McCandliss will serve as the discussant for this symposium, and in addition to synthesizing the presentations will relate these studies to his work examining the development of the multisensory brain networks that subserve reading.
A Framework for the Development of Intersensory Perception: Four Predictions of the Intersensory Redundancy Hypothesis
Robert Lickliter and Lorraine Bahrick
Infant Development Research Center
Florida International University
Intersensory redundancy is provided when the same amodal information (rhythm, tempo, duration, intensity changes) is simultaneously available and temporally synchronized across two or more sensory modalities. Most naturalistic, multimodal events provide intersensory redundancy for amodal stimulus properties as well as nonredundant modality specific information, such as the visual appearance of a face or the acoustic qualities (pitch, timbre) of a voice. What guides selective attention to these various properties of events during early development? We have proposed and provided empirical support for the Intersensory Redundancy Hypothesis (IRH), a model of selective attention comprised of a set of principles that predict and explain under what conditions perceivers attend to and process different properties of events, redundantly specified vs. nonredundantly specified. The IRH consists of four specific predictions: two predictions address the nature of selective attention to amodal vs. modality-specific properties of events and two are developmental predictions that address implications across the lifespan. We review these four predictions and discuss how they have been supported by empirical studies with both non-human animal infants and human infants. Our results suggest (1) amodal properties of events are more likely to be detected in multimodal stimulation and modality-specific properties in unimodal stimulation, (2) these processing biases lead to general salience hierarchies in attentional allocation, (3) these salience hierarchies exert a disproportionately large effect in early development, when attentional resources are most limited and cognitive load is highest, and (4) these hierarchies can continue to exert influence on later development when cognitive load and task difficulty are high. Because selective attention provides the basis for what is perceived, learned, and remembered, patterns of early selectivity can have long term, organizing effects on the development of knowledge across the life-span.
Narrowing of Intersensory Perception in Early Development
David J. Lewkowicz
Florida Atlantic University
The conventional view in developmental theory is that sensory, perceptual, and cognitive functions improve and broaden in scope as infants grow and mature. Despite this view and despite an overwhelming empirical base attesting to this fact, a small but steadily growing behavioral literature has shown that some auditory and visual perceptual functions actually decline in early life and that this is due to the effects of specific perceptual experience. Until now, the evidence of such a decline has come strictly from studies of infant response to unisensory information raising questions about the generality of this process. Recently, however, our studies have provided the first evidence of such a decline in the intersensory domain by showing that intersensory perception undergoes experience-dependent narrowing as well and, thus, that perceptual narrowing is a pan-sensory, general developmental process. I will review this new evidence by discussing the results of our studies showing that (a) 4-6 month-old infants can integrate the faces and vocalizations of a nonnative species but that infants older than 8 months of age no longer do, (b) the broad intersensory perceptual tuning first identified in young infants is present at birth, (c) intersensory perception of nonnantive audiovisual speech also narrows in infancy, and (d) the narrowing of intersensory perception is a relatively novel evolutionary innovation. I will conclude by suggesting that early perceptual experience, together with an initially unusually broad sensitivity to native as well as nonnative intersensory relations, contribute in critical ways to the development of mature intersensory perceptual expertise.
Developmental timing and the evolutionary emergence of multisensory systems in primates
Asif A. Ghazanfar
Princeton University
Understanding the evolutionary origins of multisensory systems requires an understanding of how such systems develop. The relationship between ontogenetic processes with phylogenetic ones can inform questions about whether humans and other primates share homologous behaviors and whether these behaviors are mediated by homologous neural mechanisms. All primate species tested thus far show the ability to match the faces and voices of conspecifics. This raises the question: are the developmental processes leading to the emergence of these abilities similar or different across species? The most likely answer is that because the rate of neural development in Old World monkeys and humans differs, the developmental emergence of multisensory integration probably also differs. I will present data on the comparative development of multisensory perceptual narrowing in vervet monkeys, an Old World species. The results suggest the multiple mechanisms by which developmental timing and experience can influence the formation of multisensory circuits.
Development of Multisensory Cortical and Subcortical Representations
Mark Wallace
Vanderbilt University
Multisensory circuits at various levels of the nervous system are specialized for the integration of convergent information from multiple sensory modalities. Recent work in animal models has highlighted that these circuits are not present at birth, but rather appear and mature during early postnatal life. In comparing the maturational profiles for cortical (i.e., anterior ectosylvian sulcus) and subcortical (i.e., superior colliculus) multisensory structures, there is a gradual chronological progression in the development of multisensory neurons in both structures, with cortex lagging subcortical development by several weeks on various maturational indices. Although both structures are exclusively unisensory immediately after birth, as development progresses multisensory neurons gradually appear. These first multisensory neurons differ from their adult counterparts in having very large receptive fields and lacking integrative capacity. With increasing age receptive fields become smaller and integrative capacity appears. Recent analyses further specify this developmental process by illustrating the transition in receptive field architecture from large and fairly homogenous at early ages to small and markedly heterogeneous a bit later. Together, these results highlight a strong linkage between receptive field consolidation and the appearance of integrative capacity, suggesting similar mechanistic underpinnings to these important developmental events.
A Framework for the Development of Intersensory Perception: Four Predictions of the Intersensory Redundancy Hypothesis
Robert Lickliter and Lorraine Bahrick
Infant Development Research Center
Florida International University
Intersensory redundancy is provided when the same amodal information (rhythm, tempo, duration, intensity changes) is simultaneously available and temporally synchronized across two or more sensory modalities. Most naturalistic, multimodal events provide intersensory redundancy for amodal stimulus properties as well as nonredundant modality specific information, such as the visual appearance of a face or the acoustic qualities (pitch, timbre) of a voice. What guides selective attention to these various properties of events during early development? We have proposed and provided empirical support for the Intersensory Redundancy Hypothesis (IRH), a model of selective attention comprised of a set of principles that predict and explain under what conditions perceivers attend to and process different properties of events, redundantly specified vs. nonredundantly specified. The IRH consists of four specific predictions: two predictions address the nature of selective attention to amodal vs. modality-specific properties of events and two are developmental predictions that address implications across the lifespan. We review these four predictions and discuss how they have been supported by empirical studies with both non-human animal infants and human infants. Our results suggest (1) amodal properties of events are more likely to be detected in multimodal stimulation and modality-specific properties in unimodal stimulation, (2) these processing biases lead to general salience hierarchies in attentional allocation, (3) these salience hierarchies exert a disproportionately large effect in early development, when attentional resources are most limited and cognitive load is highest, and (4) these hierarchies can continue to exert influence on later development when cognitive load and task difficulty are high. Because selective attention provides the basis for what is perceived, learned, and remembered, patterns of early selectivity can have long term, organizing effects on the development of knowledge across the life-span.
Narrowing of Intersensory Perception in Early Development
David J. Lewkowicz
Florida Atlantic University
The conventional view in developmental theory is that sensory, perceptual, and cognitive functions improve and broaden in scope as infants grow and mature. Despite this view and despite an overwhelming empirical base attesting to this fact, a small but steadily growing behavioral literature has shown that some auditory and visual perceptual functions actually decline in early life and that this is due to the effects of specific perceptual experience. Until now, the evidence of such a decline has come strictly from studies of infant response to unisensory information raising questions about the generality of this process. Recently, however, our studies have provided the first evidence of such a decline in the intersensory domain by showing that intersensory perception undergoes experience-dependent narrowing as well and, thus, that perceptual narrowing is a pan-sensory, general developmental process. I will review this new evidence by discussing the results of our studies showing that (a) 4-6 month-old infants can integrate the faces and vocalizations of a nonnative species but that infants older than 8 months of age no longer do, (b) the broad intersensory perceptual tuning first identified in young infants is present at birth, (c) intersensory perception of nonnantive audiovisual speech also narrows in infancy, and (d) the narrowing of intersensory perception is a relatively novel evolutionary innovation. I will conclude by suggesting that early perceptual experience, together with an initially unusually broad sensitivity to native as well as nonnative intersensory relations, contribute in critical ways to the development of mature intersensory perceptual expertise.
Developmental timing and the evolutionary emergence of multisensory systems in primates
Asif A. Ghazanfar
Princeton University
Understanding the evolutionary origins of multisensory systems requires an understanding of how such systems develop. The relationship between ontogenetic processes with phylogenetic ones can inform questions about whether humans and other primates share homologous behaviors and whether these behaviors are mediated by homologous neural mechanisms. All primate species tested thus far show the ability to match the faces and voices of conspecifics. This raises the question: are the developmental processes leading to the emergence of these abilities similar or different across species? The most likely answer is that because the rate of neural development in Old World monkeys and humans differs, the developmental emergence of multisensory integration probably also differs. I will present data on the comparative development of multisensory perceptual narrowing in vervet monkeys, an Old World species. The results suggest the multiple mechanisms by which developmental timing and experience can influence the formation of multisensory circuits.
Development of Multisensory Cortical and Subcortical Representations
Mark Wallace
Vanderbilt University
Multisensory circuits at various levels of the nervous system are specialized for the integration of convergent information from multiple sensory modalities. Recent work in animal models has highlighted that these circuits are not present at birth, but rather appear and mature during early postnatal life. In comparing the maturational profiles for cortical (i.e., anterior ectosylvian sulcus) and subcortical (i.e., superior colliculus) multisensory structures, there is a gradual chronological progression in the development of multisensory neurons in both structures, with cortex lagging subcortical development by several weeks on various maturational indices. Although both structures are exclusively unisensory immediately after birth, as development progresses multisensory neurons gradually appear. These first multisensory neurons differ from their adult counterparts in having very large receptive fields and lacking integrative capacity. With increasing age receptive fields become smaller and integrative capacity appears. Recent analyses further specify this developmental process by illustrating the transition in receptive field architecture from large and fairly homogenous at early ages to small and markedly heterogeneous a bit later. Together, these results highlight a strong linkage between receptive field consolidation and the appearance of integrative capacity, suggesting similar mechanistic underpinnings to these important developmental events.