Cross-Modal Perception in the Framework of Non-Riemannian Sensory Space
Last modified: 2011-11-16
Abstract
Though human sensations, such as the senses of hearing, sight, etc., are independent each other, the interference between two of them is sometimes observed, and is called cross-modal perception[1]. Hitherto we studied unimodal perception of visual sensation[2] and auditory sensation[3] respectively by differential geometry[4]. We interpreted the parallel alley and the distance alley as two geodesics under different conditions in a visual space, and depicted the trace of continuous vowel speech as the geodesics through phonemes on a vowel plane.
In this work, cross-modal perception is similarly treated from the standpoint of non-Riemannian geometry, where each axis of a cross-modal sensory space represents unimodal sensation. The geometry allows us to treat asymmetric metric tensor and hence a non-Euclidean concept of anholonomic objects, representing unidirectional property of cross-modal perception. The McGurk effect in audiovisual perception[5] and ‘rubber hand’ illusion in visual tactile perception[6] can afford experimental evidence of torsion tensor. The origin of 'bouncing balls' illusion[7] is discussed from the standpoint of an audiovisual cross-modal sensory space in a qualitative manner.
In this work, cross-modal perception is similarly treated from the standpoint of non-Riemannian geometry, where each axis of a cross-modal sensory space represents unimodal sensation. The geometry allows us to treat asymmetric metric tensor and hence a non-Euclidean concept of anholonomic objects, representing unidirectional property of cross-modal perception. The McGurk effect in audiovisual perception[5] and ‘rubber hand’ illusion in visual tactile perception[6] can afford experimental evidence of torsion tensor. The origin of 'bouncing balls' illusion[7] is discussed from the standpoint of an audiovisual cross-modal sensory space in a qualitative manner.
References
[1] G.A.Calvert, C.Spence and B.E.Stein, eds., The Handbook of Multisensory Processes. The MIT Press, 2004. [2] M.Shimbo, T.Yamanoi and M.Kawaguchi, On a Hypothesis in Luneburg’s Theory of Binocular Space (in Japanese). Bulletin of the Faculty of Engineering, Hokkaido University, 87(1978), 169-173. [3] J.Toyama and M.Shimbo, Phoneme Discrimination Ellipses on a Projective Vowel Plane and the Vowel Glides Calculated Thereby. Tensor, N.S., 51, 1(1992), 85-92. [4] J.A.Schouten, Ricci-Calculus, 2nd ed. Springer-Verlag, 1954. [5] H.McGurk and J.MacDonald, Hearing Lips and Seeing Voices. Nature, 264(1976), 746-748. [6] M.Botvinick, Probing the Neural Basis of Body Ownership. Science, 305(2004), 782-783. [7] R.Sekuler, A.B.Sekuler and R.Lau, Sound Alters Visual Motion. Nature, 385(1997), 308.