Integration of iconic gestures and speech in left superior temporal areas 
boosts speech comprehension under adverse listening conditions

Holle, Henning; Obleser, Jonas; Rueschemeyer, Shirley-Ann; Gunter, Thomas C.

doi:10.1016/j.neuroimage.2009.08.058

アイテム詳細

登録内容を編集ファイル形式で保存

一時保存へ追加

タグ情報を表示リリース履歴を表示詳細要約

公開

学術論文

Integration of iconic gestures and speech in left superior temporal areas boosts speech comprehension under adverse listening conditions

MPS-Authors

/persons/resource/persons19902

Obleser, Jonas
Department Neuropsychology, MPI for Human Cognitive and Brain Sciences, Max Planck Society;

/persons/resource/persons19687

Gunter, Thomas C.
Department Neuropsychology, MPI for Human Cognitive and Brain Sciences, Max Planck Society;

External Resource

There are no locators available

Fulltext (restricted access)

There are currently no full texts shared for your IP range.

フルテキスト (公開)

Holle_2010.pdf
(プレプリント), 2MB

付随資料 (公開)

There is no public supplementary material available

引用

Holle, H., Obleser, J., Rueschemeyer, S.-A., & Gunter, T. C. (2010). Integration of iconic gestures and speech in left superior temporal areas boosts speech comprehension under adverse listening conditions. NeuroImage, 49(1), 875-884. doi:10.1016/j.neuroimage.2009.08.058.

引用: https://hdl.handle.net/11858/00-001M-0000-0010-BDEE-0

要旨

Iconic gestures are spontaneous hand movements that illustrate certain contents of speech and, as such, are an important part of face-to-face communication. This experiment targets the brain bases of how iconic gestures and speech are integrated during comprehension. Areas of integration were identified on the basis of two classic properties of multimodal integration, bimodal enhancement and inverse effectiveness (i.e., greater enhancement for unimodally least effective stimuli). Participants underwent fMRI while being presented with videos of gesture-supported sentences as well as their unimodal components, which allowed us to identify areas showing bimodal enhancement. Additionally, we manipulated the signal-to-noise ratio of speech (either moderate or good) to probe for integration areas exhibiting the inverse effectiveness property. Bimodal enhancement was found at the posterior end of the superior temporal sulcus and adjacent superior temporal gyrus (pSTS/STG) in both hemispheres, indicating that the integration of iconic gestures and speech takes place in these areas. Furthermore, we found that the left pSTS/STG specifically showed a pattern of inverse effectiveness, i.e., the neural enhancement for bimodal stimulation was greater under adverse listening conditions. This indicates that activity in this area is boosted when an iconic gesture accompanies an utterance that is otherwise difficult to comprehend. The neural response paralleled the behavioral data observed. The present data extends results from previous gesture-speech integration studies in showing that pSTS/STG plays a key role in the facilitation of speech comprehension through simultaneous gestural input. Crown Copyright © 2009.