Item

Abstract

Affective information constitutes a key feature of audiovisual speech and plays a fundamental role in everyday communication and consequent social interactions. In spite of the perceptual saliency of affective information, however, its role in the audiovisual integration of speech has not, as yet, received much attention from researchers interested in multisensory perception. Here, we investigated the effects of affective content on audiovisual speech integration. Participants were presented with pairs of spatially discrepant auditory and visual speech stimuli conveying various emotional states. To measure the impact of affective valence on multisensory integration, participants were asked to report the relative spatial location of the auditory speech with respect to that of the visual, speaking face.
In Experiment 1, the reliability of participants‘ estimates was lower for both angry and fearful audiovisual speech stimuli compared to emotionally neutral stimuli. In Experiment 2, the reliability of estimates was lower for both fearful and happy stimuli compared to neutral. Reduced reliability of perceptual estimates regarding intersensory conflict indicates a stronger coupling between unisensory signals. Therefore, these results imply a stronger coupling of audiovisual stimuli sharing positive or negative emotional tone, and provide the first psychophysical evidence for the role of affective information in multisensory integration.