hide
Free keywords:
-
Abstract:
In our daily life, we are often confronted with objects that give rise to signals in multiple sensory modalities. The brain's challenge is to integrate information from multiple
senses into a unified percept. Combining psychophysics and
fMRI, the present study investigates how task-irrelevant
object source sounds affect category-selective activations for
visual faces in the fusiform and landmarks in the parahippocampal
gyri. In a visual selective attention paradigm,
subjects categorized degraded object pictures as landmarks or
faces while ignoring intact object source sounds that were
either semantically congruent or incongruent. The 2X2
factorial design manipulated (i) Visual category: Animal vs.
Landmark, and (ii) Auditory category: Animal vocalization
vs. Sound associated with Landmark. Behaviorally, incongruent
trials were associated with longer response times. This
effect emerged due to an interference of incongruent sounds
associated with landmarks on the categorization of visual
faces. At the neural level, only landmark-selective activation
in the parahippocampal gyrus, but not face selective activations
in the fusiform gyrus were modulated by the congruency
of an irrelevant object sound. More specifically, activations in
the anterior parahippocampal gyrus showed additive effects of
both visual and auditory category information. Effective
connectivity analysis indicated that parahippocampal
responses are amplified by incongruent auditory inputs via
enhanced coupling between auditory and occipito-temporal
cortices. Collectively, these results suggest that a region in the
parahippocampal gyrus integrates information about object
categories from multiple senses. In contrast, face-selective
responses in fusiform gyrus are more robust and less
influenced by task-irrelevant sounds, even in the context of
a behavioural audiovisual interference effect.
