Perceived congruence between the changing patterns of a visual image and pitch of a sound
Last modified: 2011-09-02
Abstract
Perceived congruence between the changing patterns of a visual image and pitch of a sound was examined by a series of rating experiments.
Vertical correspondence of direction between the movement of an image and the pitch shift of a sound had a strong effect on the perceived congruence; the combination of a rising image and an ascending pitch, and that of a falling image and a descending pitch created higher congruence than the alternative combinations.
The combination of an enlarging image and an ascending pitch, and that of a　reducing image and a descending pitch also created higher congruence.
Furthermore, the combination of a movement sliding from left to right and an ascending pitch, and that of a movement sliding from right to left and a descending pitch created higher congruence.
When the visual transformation pattern was a complex of these three kinds of movements, the vertical direction transformation was the most effective factor in determining congruent pitch pattern. The next most effective transformation factor was the enlarging and reducing image. The transformation of right and left direction movement was not strongly effective in determining the perceived congruence between complex visual movements and pitch patterns
Vertical correspondence of direction between the movement of an image and the pitch shift of a sound had a strong effect on the perceived congruence; the combination of a rising image and an ascending pitch, and that of a falling image and a descending pitch created higher congruence than the alternative combinations.
The combination of an enlarging image and an ascending pitch, and that of a　reducing image and a descending pitch also created higher congruence.
Furthermore, the combination of a movement sliding from left to right and an ascending pitch, and that of a movement sliding from right to left and a descending pitch created higher congruence.
When the visual transformation pattern was a complex of these three kinds of movements, the vertical direction transformation was the most effective factor in determining congruent pitch pattern. The next most effective transformation factor was the enlarging and reducing image. The transformation of right and left direction movement was not strongly effective in determining the perceived congruence between complex visual movements and pitch patterns
References
Eitan, Z. & Granot, R. Y. (2006). How music moves: Musical parameters and listeners’ images of motion. Music Percept., 23, 221-248.
Iwamiya, S. (2008). Subjective congruence between moving picture and sound. Proc. 10th Int. Conf. Music Perception and Cognition, Sapporo, 83-87.
Lipscomb, S. D. & Kim, E. M. (2004). Perceived match between visual parameters and auditory correlates: An experimental multimedia investigation. Proc. 8th Int. Conf. Music Perception and Cognition, Evanston, 72-75.
Iwamiya, S. (2008). Subjective congruence between moving picture and sound. Proc. 10th Int. Conf. Music Perception and Cognition, Sapporo, 83-87.
Lipscomb, S. D. & Kim, E. M. (2004). Perceived match between visual parameters and auditory correlates: An experimental multimedia investigation. Proc. 8th Int. Conf. Music Perception and Cognition, Evanston, 72-75.