English
 
User Manual Privacy Policy Disclaimer Contact us
  Advanced SearchBrowse

Item

ITEM ACTIONSEXPORT

Released

Conference Paper

Perception of Prominence Intensity in audio-visual Speech

MPS-Authors
/persons/resource/persons84115

Nusseck,  M
Department Human Perception, Cognition and Action, Max Planck Institute for Biological Cybernetics, Max Planck Society;
Max Planck Institute for Biological Cybernetics, Max Planck Society;

/persons/resource/persons84298

Wallraven,  C
Department Human Perception, Cognition and Action, Max Planck Institute for Biological Cybernetics, Max Planck Society;
Max Planck Institute for Biological Cybernetics, Max Planck Society;

External Ressource
Fulltext (public)

AVSP-2007-Nusseck.pdf
(Any fulltext), 160KB

Supplementary Material (public)
There is no public supplementary material available
Citation

Nusseck, M., Cunningham, D., Ruiter, J., & Wallraven, C. (2007). Perception of Prominence Intensity in audio-visual Speech. In J. Vroomem, M. Swerts, & E. Krahmer (Eds.), International Conference on Auditory-Visual Speech Processing 2007 (AVSP 2007) (pp. 1-6).


Cite as: http://hdl.handle.net/11858/00-001M-0000-0013-CBF7-1
Abstract
Multimodal prosody carries a wide variety of information Here, we investigated the roles of visual and the auditory information in the production and perception of different emphasis intensities. In a series of video recordings, the intensity, location, and syntactic category of the emphasized word were varied. Physical analyses demonstrated that each speaker produced different emphasis intensities, with a high degree of individual variation in information distribution. In the first psychophysical experiment, observers easily distinguished between the different intensities. Interestingly, the pattern of perceived intensity was remarkably similar across speakers, despite the individual variations in the use of different visual and acoustic modalities. The second experiment presented the recordings visually, acoustically, and audiovisually. Overall, while the audio only condition was very similar to the audiovisual condition, there was a clear influence of visual information. Weak visual information lead to a weaker audiovisual intensity, while stong visual information enhanced audiovisual intensity.