The integration of sound and vision is an essential aspect of coherent perception. Salient peripheral free-field sounds improve processing of subsequent visual stimuli that appear at the same site as the sound. However, it is still unclear at what level of processing sounds affect visual perception. We investigated whether sound cues with interaural time differences are sufficient to modulate the perception of upcoming visual targets. Visual targets following sound cues were presented at several horizontal eccentricities. With a short cue-target onset asynchrony, subjects discriminated oriented visual stimuli more accurately at visual field locations that corresponded to the interaural time difference of the preceding sound. Interestingly, visual discrimination at nearby visual field locations remained unaffected by sounds. With long cue-target delays, visual discrimination performance decreased at the cued location but not at nearby locations. Our results suggest direct associations between auditory maps representing interaural time differences and corresponding sites in visual field maps.