A growing number of studies suggest that speech perception is influenced by the consistency of the coupling between the auditory (phonological) and the visual (orthographic) forms of words (for a review, see Ziegler et al., 2008, J Exp Psych: Learn Mem Cog). Spoken words with inconsistent rimes (that can be spelled in multiple ways, e.g. the rime /-ANE/ is spelled differently in /BRAIN/ and in /SANE/) take more time to be processed than spoken words with consistent rimes (that can be spelled only one way, like /-ABE/). This auditory consistency effect could either be due to online activation of orthographic word forms in the left posterior cortex (BA 37 or 39/40) or to the direct contamination of the speech areas (BA 22, 44, 45) by orthographic knowledge, which would take place during reading and writing acquisition. To disentangle these two hypotheses, we conducted a bimodal fMRI experiment. Orthographic decisions on visual words involved left BA 22 and 45. Auditory lexical decisions on consistent and inconsistent words involved a wide left temporo-frontal network. The consistency effect (inconsistent minus consistent words) was located in left BA 45. These results suggest that orthographic knowledge contaminates speech perception during the process of learning how to read and write.