We studied how inconsistency between directions in audio-visual motion does effect on the auditory processing. Using a whole-head magnetoencephalography we localized the sources of activity in the primary auditory cortex. We found that when the direction of audio and visual motion was opposite the auditory N1 component (about 100 ms after stimulus onset) had larger amplitude than when the direction was the same. Such an early effect of the audio-visual inconsistency observed in the auditory cortex indicates a starting time point of audio-visual integration which happens in primary sensory areas.