To what extent does our prior experience of particular combinations of audiovisual stimuli influence how we subsequently bind those stimuli? In the present study, we addressed this question by testing English and Spanish speakers (having little prior experience of Spanish and English, respectively) on a simultaneity judgment (SJ) task involving the presentation of visual and auditory channels of English or Spanish spoken sentences, that had been desynchronized by varying amounts. The results revealed that for English observers, the visual speech stream had to lead the auditory speech stream by a significantly larger interval in English than in Spanish for simultaneity to be perceived. By contrast, the Spanish observers showed the opposite pattern of results. We propose that this modulation of multisensory perception by the observer's prior experience is a consequence of the constraining role that visual information plays in the temporal alignment of audiovisual speech signals.