The synaesthetic congruency between auditory and visual stimuli has been shown to modulate people’s performance in speeded classification tasks, in which they have to classify stimuli in one modality whilst simultaneously trying to ignore concurrent stimuli presented in another (irrelevant) modality. In this study, we used psychophysical procedures to demonstrate that the association between auditory pitch and visual size can influence audiovisual integration in the temporal domain. In Experiment 1, the implicit association test (IAT) was used to verify the synaesthetic association between the proposed audiovisual dimensions. In Experiment 2, we investigated whether these synaesthetic associations would modulate the precision of participants’ temporal order judgments (TOJs) in a temporal ventriloquism task. Participants judged the temporal order of the stimuli more accurately on those trials where the temporally proximate auditory and visual stimuli were synaesthetically congruent, than on those trials where there was no such congruency. These results provide the first empirical evidence that audiovisual temporal integration is modulated by the synaesthetic congruence between auditory and visual stimuli.