Phonetic symbolism is a phenomenon that speech sound evokes images based on sensory experiences and thus often discussed with the similarity for cross-modal correspondence. By using Garner’s task, Hirata, Kita, and Ukita (2009) showed the cross-modal congruence between brightness and voiced/voiceless consonants in Japanese speech sound, which is known as phonetic symbolism. In the present study, we examined the effect of the meaning of mimetics (lexical words which has relationship between its sound and meaning, like “ding-dong”) in Japanese language on the cross-modal correspondence. We conducted an experiment with Chinese speech sounds with or without aspiration and the same task to Chinese people. Chinese vocabulary also contains mimetics but the existence of aspiration doesn’t relate to the meaning of Chinese mimetics. As a result, Chinese speech sounds with aspiration, which resemble voiceless consonants, were matched with white color, whereas those without aspiration were matched with black. This result is identical to its pattern in Japanese people and consequently suggests that cross-modal correspondence occurs without the effect of the meaning of mimetics. The problem that whether these cross-modal correspondences are purely based on physical properties of speech sound or affected from phonetic properties remains for further study.