Searching for the talking head: The cocktail party revisited
Agnès Alsius, Salvador Soto-Faraco
Poster
Last modified: 2008-05-15
Abstract
Everyday life environments are characterized by a continuous flow of information arriving to our brains from different senses. A crucial question in multisensory research is how all these pieces of sensory information are organized, selected and integrated in a coherent representation of the world. Some authors have claimed that correlated multisensory information is bound pre-attentively, whereas other findings indicate that attention might play a crucial role during binding. Here, we used a visual (Experiments 1-4) and auditory (Experiments 5 & 6) search paradigm to explore whether audiovisual matching of natural stimuli (i.e., talking faces) occurs prior to the allocation of spatial selective attention. In Experiments 1-4, silent video-clips of four faces simultaneously speaking were presented along with one single auditory message. Participants were required to search for the face whose lips corresponded to the auditory sentence (we used both, a localization and a detection task). Our results showed that RT and error rates increased linearly with display size (the number of talking faces), thus indicating serial search and challenging the idea that audiovisual matching occurs prior to the allocation of visual spatial attention. In Experiments 5-6, we used an auditory search task with four different auditory sentences played from different spatial locations along with a single central talking face. Importantly, the results of this experiment showed that, while in the localization task RT increased linearly with the number of attended elements, the detection task showed a flat slope, consistent with a parallel search mechanism. On the basis of these findings, we suggest that spatial selective attention can play a crucial role in audiovisual binding in natural environments, and that it plays a differential role in vision and in audition.