It is uncertain whether optimal multisensory integration (see Ernst & Bülthoff, 2004) occurs in a purely perceptual bottom-up manner, or whether they also require executive resources to function adequately. If executive resources are required for optimal integration, then it should be possible to observe effects of secondary task performance and/or working memory load (WML) on the relative weighting of the senses under conditions of intersensory conflict. We examined the effect of a secondary task and WML on the relative weighting of visual and proprioceptive cues to hand location prior to reaching for a visual target. Visual and proprioceptive cues to the position of the right hand were put into varying degrees of conflict in the azimuthal plane and in radial depth by displacing the hand relative to an illusory visual image of it (provided by a mirror placed at the midline, facing the left hand; Holmes et al., 2004). The secondary task required participants to remember a string of digits whilst observing the (illusory) right hand under varying conditions of conflict and then reaching for a visual target of that hand. There were three conditions of load (No secondary task / Low WML; the string to be remembered was 01234 / High WML; the string to be remembered was a random series - e.g. 02143). Fig. 1 shows that WML increased visual capture by the illusion, regardless of the direction of visual-proprioceptive conflict (F(1,14)=3.6, p<.05). This effect of WML was driven by a significant difference in visual capture between the No WML condition and the Low WML condition – indicating that the executive demands of task-switching (rather than WML load) drives this effect. We conclude that, in this case, optimal multisensory integration requires executive resources.

Ernst, M.O. and Bülthoff, H.H. (2004) Merging the senses into a robust percept. Trends Cogn. Sci. 8, 162-169
Holmes, N.P. et al. (2004) When mirrors lie: “Visual capture� of arm position impairs reaching performance. Cogn. Affective Behav. Neurosci. 4, 193-200

Fig. 1: Mean reaching locations in x and y coordinates across working memory conditions. Y indicates radial dept from the participant. X indicates the azimuthal dimension. In all target conditions proprioception was veridical. When reaching for the central target there was no visual-proprioceptive conflict. When reaching for the far and near targets visual information was displaced relative to proprioception in the degree of radial depth indicated by target locations. When reaching for the left and right targets, visual information was displaced relative to proprioception in the azimuthal direction indicated by target locations.