Sensory substitution devices provide through an unusual sensory modality (the substituting modality, for example audition) the kind of information that is normally accessed through another sensory modality (the substituted modality, for example vision). Since their inception in the 60’s various kinds of devices have been developed, tested, and shown to allow their users to behave to some degree as if they possessed the substituted sensory organ. These systems thus question the usual taxonomy of our sensory modalities and the characterisation of a perceptual experience. Through a set of behavioural studies involving localisation, recognition, and categorisation tasks with visual-to-auditory substitution devices, we addressed the question of which sensory modality the acquired perception belongs to. Though certain results might be taken to point to the conclusion that perception with a sensory substitution device belongs to the substituted modality, overall evidence leads to an alternative view on sensory substitution. According to it, the experience after sensory substitution is a transformation, extension, or augmentation of our perceptual capacities, rather than something equivalent or reducible to an already existing sensory modality.