Previous studies have shown that vibratory stimuli can convey texture information in both the tactile and auditory systems. It has also been shown that auditory stimuli can influence tactile texture perception. However, how the auditory and tactile systems process the vibratory stimuli as textures is unclear. In this study, we examined, 1) the similarities between tactile and auditory roughness perception using the same vibratory stimuli, and 2) how unattended auditory stimuli influence roughness perception of tactile stimuli. We first obtained the texture-based vibratory stimuli by recording through a stationary probe placed against moving textures. These stimuli were then used in three experimental conditions. In condition 1 (Tactile-only), participants were presented only with vibrotactile stimuli. In condition 2 (Auditory-only), participants were presented with auditory stimuli alone. In condition 3 (Audio-Tactile), participants were presented with combined auditory and tactile stimuli. Tactile-only and Audio-Tactile conditions required participants to rate the subjective-magnitude estimates of roughness of the tactile stimuli, while the auditory-only condition required participants to rate the perceived auditory roughness. We found a correlation between tactile and auditory ratings of roughness. In addition, we found that auditory stimuli increased perceived tactile roughness ratings. The correlation between perceived tactile roughness and perceived auditory roughness indicates that these sensory systems may use similar but not identical texture information processing mechanisms.