We investigated whether perceiving predictable “ups and downs” in acoustic pitch (as can be heard in musical melodies) can influence the spatial processing of visual stimuli as a consequence of a ‘spatial recoding’ of sound (see Foster & Zatorre, 2010, Rusconi et al., 2006). Event-related potentials (ERPs) were recorded while participants performed a color discrimination task of a visual target that could appear either above or below a centrally-presented fixation point. Each experimental trial started with an auditory isochronous stream of 11 tones including a high- and a low-pitched tone. The visual target appeared isochronously after the last tone. In the ‘non-predictive’ condition, the tones were presented in an erratic fashion (e.g., “high-low-low-high-high-low-high…”). In the ‘predictive condition’, the melodic combination of high- and low-pitched tones was highly predictable (e.g., “low-high-low-high-low …”). Within the predictive condition, the visual stimuli appeared congruently or incongruently with respect to the melody (“…low-high-low-high-low-UP” or “…low-high-low-high-low-DOWN”, respectively).
Participants showed faster responses when the visual target appeared after a predictive melody. Electrophysiologically, early (25-150ms) amplitude effects of predictability were observed in frontal and parietal regions, spreading to central regions (N1) afterwards. Predictability effects were also found in the P2-N2 complex and the P3 in central and parietal regions. Significant auditory-to-visual congruency effects were also observed in the parieto-occipital P3 component.
Our findings reveal the existence of crossmodal effects of perceiving auditory isochronous melodies on visual temporal orienting. More importantly, our results suggest that pitch information can be transformed into a spatial code that shapes the spatial processing in other modalities such as vision.