Auditory-Visual Interactions and Rhythmic Structure

Victoria Cheah, Fruzsina Soltesz, Denes Szucs, Usha Goswami
Poster
Time: 2009-06-30  09:00 AM – 10:30 AM
Last modified: 2009-06-04

Abstract


Many meaningful auditory stimuli in the natural world have a rhythmic structure, for example the beats in music and stress patterns in speech. Even infants are able to detect these auditory patterns and it is thought that rhythmic regularities of stress and prosody in speech play an important role in infants’ early language acquisition. We are interested in whether rhythmic structure may have a broadly organizing role across modalities, and what the hallmarks of such organization are at the behavioural level. In this task, adult participants were asked to detect a brief visual flash that was presented either within a strong auditory rhythmic context, or without a rhythmic context. Rhythmic auditory beats were presented at a rate of 1.67 Hz (600 ms period) and visual flashes were presented in alternate auditory periods with ISIs varying randomly between 20 ms and 500 ms in 20 ms steps. In a unimodal baseline task, visual flashes were presented with the same temporal distribution, but in the absence of any background auditory rhythm. Participants’ reaction times to the flash for each of the 25 ISI values were plotted to create a reaction time profile across the whole auditory period. The pattern of this profile was compared in bimodal and unimodal conditions.

We report two major findings. Firstly, facilitatory effects in the bimodal condition were observed, but these were only significant toward the end of the auditory period. That is, compared to the visual-only baseline, there was no net facilitatory effect observed when the flash immediately followed the sound. Reaction times to the flash were only significantly faster at longer lags when the next sound was imminent, which may suggest the presence of anticipatory effects based on estimation of the auditory rhythmic interval. Secondly, there were differences in the reaction time profile pattern of the two conditions. Both conditions were characterized by a decreasing trend in reaction time across the auditory period. However, we also observed small but regular periodic patterns of fluctuation in reaction time throughout the auditory period. These regular patterns were present in the bimodal condition but not in the unimodal condition. Since neural oscillatory activity has been shown to correlate with behavioural reaction times both in animal (Lakatos et al, 2008) and human studies (Senkowski et al, 2006), it would be interesting to test whether the temporal pattern of effects observed here are related to neuronal oscillatory activity.

In summary, our results indicate that auditory facilitation of visual detection shows a temporal dependency on rhythmic period, and the pattern of this temporal dependency shows evidence of more complex organization. Secondly, our data suggest that temporal rhythm may have a wide-ranging influence across sensory systems. That is, temporal regularity in the auditory modality may structure motor responses to irregular input arriving through the visual modality.

Conference System by Open Conference Systems & MohSho Interactive Multimedia