Many studies have previously reported on the effects of training or expertise learning on the visual perception of highly similar objects such as faces, letters and textures (e.g. Schwartz et al, PNAS, 2002). However, very little is known about whether training effects are specific to the training modality only or whether these effects can transfer to a non-trained modality. Moreover, the neural substrates underlying the effects of training within and across modalities have previously not been investigated. We conducted two studies in which we tested the effects of perceptual training within and across modalities. Specifically, participants were trained to categorise a set of fabric samples as either ‘natural’ or not and we measured behavioural performance (Experiment 1) and neural activation (Experiment 2) as a consequence. The stimuli consisted of 32 different fabrics which varied gradually from natural to synthetic. During the learning session these stimuli were pseudorandomly divided into two subsets, one subset was learnt using vision and the other using touch. In Experiment 1, participants conducted a baseline block in which they were required to categorise each of the stimuli as natural or synthetic using either vision or touch. Following baseline, participants were then trained on half of the stimuli through vision only and the other half through touch only. At the end of the training session, categorisation performance was measured both in the same modality as the training modality and in the other modality. We found effects of training within modalities with a significant increase in categorisation accuracy after training. In the cross-modal conditions, training effects were found to transfer from vision to touch but not from touch to vision. In other words, whereas visual performance benefitted from tactile training of the texture stimuli, performance in the tactile modality did not benefit from visual training. In Experiment 2, functional data was acquired while participants performed a visual categorization task before and after training. Between scanning sessions the participants were trained to categorise the stimuli using either vision or touch. Preliminary analysis of the functional data comparing brain activity before and after the training session revealed differences not only in early sensory regions, but also in multisensory and higher cognitive region, particularly for the subset of stimuli learned by touch. Our results show that there are cross-modal effects during perceptual learning although these effects seem to be unidirectional and dominated by visual processing.