Content and temporal cues have been shown to interact during audiovisual (AV) speech identification. Typically, the most reliable unimodal cue is used to identify specific speech features; however, visual cues are only used if the audiovisual stimuli are presented within a certain temporal integration window (TWI). This suggests that temporal cues denote whether unimodal stimuli belong together and should be integrated. It is unknown whether temporal cues also provide information about speech content. Since spoken syllables have naturally varying audiovisual onset asynchronies, we hypothesize that for suboptimal AV cues presented within the TWI, these natural AV onset differences can aid in speech identification. To test this, we presented low-intensity auditory syllables concurrently with visual speech signals, and varied the stimulus onset asynchronies (SOA) of the audiovisual pair, while participants identified the syllables. We revealed that the most reliable cues of the audiovisual input were used to identify specific speech features (e.g. voicing). Additionally, we showed that the TWI was wider for congruent stimulus pairs. Finally, we found a specific response pattern across the SOA-range for syllables that were not reliably identified by the unimodal cues, which we explained by the use of natural onset differences between audiovisual speech signals. This indicates that temporal cues not only determine whether or not different inputs belong together, but additionally convey identity information of audiovisual pairs. These results provide a detailed behavioral basis for further neuro-imaging and stimulation studies to unravel the neurofunctional mechanisms of the audio-visual-temporal interplay within speech perception.

stimulus onset asynchrony; audiovisual speech; temporal relationship