Extinction is diagnosed when patients respond to a single contralesional item but fail to detect this item when an ipsilesional item is present concurrently. It is considered to be a disorder of attention characterized by a striking bias for the ipsilesional stimulus at the expense of the contralesional stimulus. Extinction has been studied mainly in the visual modality but it occurs also in other sensory modalities (touch, audition) and hence can be considered a multisensory phenomenon. The functional and neuroanatomical relations between extinction in different modalities are poorly understood. It could be hypothesised that extinction deficits in different modalities emerge after damage to both common (attention specific) and distinct (modality specific) brain regions. Here, we used voxel-based mophometry to examine the neuronal substrates of visual versus tactile extinction in a large group of stroke patients (n=454). We found that extinction deficits in the two modalities were significantly correlated (r=0.341; p<0.01). Lesions to inferior parietal lobule and middle frontal gyrus were linked to visual extinction, while lesions involving the superior temporal gyrus were associated with tactile extinction. Damage within the middle temporal gyrus was linked to both types of deficits but interestingly these lesions extended into the middle occipital gyrus in patients with visual but not tactile extinction. White matter damage within the temporal lobe was associated with both types of deficits, including lesions within long association pathways involved in spatial attention. Our findings indicate both common and distinct neural mechanisms of visual and tactile extinction.