Cerebral encoding of structures from different sensory sources

Corinne Jola

Last modified: 2013-05-05

Abstract


Recent neuroimaging studies have investigated cortical activity in response to passive exposure to long segments of visual and/or auditory stimuli (eg Hasson et al. 2004; Jola et al. 2013; Speer et al. 2007). The transition from short to long stimulus material enables studying cognitive processes that are closer to real life. Moreover, processes involved in perception of structure in time do ultimately require stimuli of extended duration. Despite the challenges that such complex but ecologically more valid stimuli pose, the results are promising. I will thus present recent findings of how long stimuli are processed in the case of dance. Dance is a multisensory stimulus; as it is generally performed to music. We found evidence of enhanced synchronisation of neuronal activity in participants’ frontal brain areas when the dance recordings were edited (and thus more directed) compared to when they were presented as a single shot recording. This is in support of the editing hypothesis, namely, that visually presented stimuli require directing spectators’ attention in order to elicit enhanced shared responses (see Jola et al. 2013). Yet a number of additional studies showed that spectators perceive dance structures in very individual ways (eg Pollick et al. 2012; deLahunta & Barnard 2005). However, this heterogeneity is reduced if the spectators have a shared practice: we found evidence for enhanced synchronisation in sensorimotor and frontal areas in response to watching a single shot dance video, under the condition that the participants have physically learned the movement before they passively observed them in the scanner (Jola et al. in prep). This finding suggests that for shared motor resonance, a predisposition of a consolidated structure is important. On the premise of a multisensory structure encoding mechanism, we thus currently investigate the cerebral encoding of dance structures in analogy to language and music processing (Pallier et al. 2011). I will outline the experimental challenges an ecologically valid study of the moving (ie dancing) human body poses and present our preliminary results.

Keywords


action observation; fMRI; audiovisual composition

References


Hasson U, Nir Y, Levy I, Fuhrmann G, Malach R (2004) Intersubject synchronization of cortical activity during natural vision. Science 303: 1634-40.

Herbec A, Kauppi JP, Jola C, Haodan T, Tohka J, Pollick FE (in prep) Inter-subject synchronization while viewing edited versus non-edited videos of a dance performance.

Jola C, McAleer P, Grosbras MH, Love, SA, Morison G, Pollick FE (2013-in press) Uni- and multisensory brain areas are synchronised across spectators when watching unedited dance. i-Perception. dx.doi.org/10.1068/i0536IMRF

Jola C, Gazzola V, Keysers Ch (in prep). Synchronised brain activity during action observation across expert spectators.

Katie N, Glowinski D, Murphy H, Jola C, McAleer P, Darshane N, Penfield K, Camurri A, Pollick FE (in prep) Brain mechanisms of action understanding revealed by watching extended dance solos.

De Lahunta S, Barnard P (2005) What’s in a phrase? In Tanz im Kopf, J Birringer & J Fenger (Eds). Hamburg LIT Verlag.

Pallier C, Devauchelle AD, Dehaene S (2011) Cortical representation of the constituent structure of sentences. Proc Natl Acad Sci 108:2522-7.

Pollick F, Noble K, Darshane N, Murphy H, Glowinski D, McAleer P, Jola C, Penfield K, Camurri A (2012). Using a novel motion index to study the neural basis of event segmentation. Conference Proceedings, i-Perception, 4: 225.

Reason M, Reynolds D, Kay R, Jola C, Grobras M-H, Pollick FE (in prep) Spectators’ aesthetic experience of sound and movement in dance performance: a multi-methodological investigation.

Speer NK, Zacks JM, Reynolds JR (2007) Human brain activity time-locked to narrative event boundaries. Psychol Sci 18: 449-55.

Conference System by Open Conference Systems & MohSho Interactive