We examined the novel crossmodal semantic congruency effect on bistable figures in which a static stimulus gives rise to two competing percepts that alternate over time. Participants viewed the bistable figure "my wife or my mother-in-law" while listening to the voice of an old woman or a young lady speaking in an unfamiliar language. They had to report whether they saw the old woman, the young lady, or a mixed percept. Robust crossmodal semantic congruency effects in the measures of the first percept and the predominance duration were observed. The possibilities that the participants simply responded to, and/or that they fixed at the location in favor of, the percept congruent with the sound that they happened to hear were ruled out. When the participants were instructed to maintain their attention to a specific view, a strong top-down modulation on the perception of bistable figure was observed, although the audiovisual semantic congruency effect still remained. These results thus demonstrate that top-down attention (i.e., selection and/or voluntary control) modulates the audiovisual semantic congruency effect. As the alternating percepts in bistable figures indicate competition for conscious perception, this study has important implications for the multifaceted interactions between multisensory processing, attention, and consciousness.