English
 
User Manual Privacy Policy Disclaimer Contact us
  Advanced SearchBrowse

Item

ITEM ACTIONSEXPORT
  Multimodality and the origin of a novel communication system in face-to-face interaction

Macuch Silva, V., Holler, J., Ozyurek, A., & Roberts, S. G. (2020). Multimodality and the origin of a novel communication system in face-to-face interaction. Royal Society Open Science, 7: 182056. doi:10.1098/rsos.182056.

Item is

Basic

show hide
Item Permalink: http://hdl.handle.net/21.11116/0000-0005-4615-5 Version Permalink: http://hdl.handle.net/21.11116/0000-0005-7BEE-6
Genre: Journal Article

Files

show Files
hide Files
:
rsos.182056.pdf (Publisher version), 2MB
Name:
rsos.182056.pdf
Description:
-
Visibility:
Public
MIME-Type / Checksum:
application/pdf / [MD5]
Technical Metadata:
Copyright Date:
2020
Copyright Info:
The Authors. Published by the Royal Society under the terms of the Creative Commons Attribution License http://creativecommons.org/licenses/by/4.0/, which permits unrestricted use, provided the original author and source are credited.

Locators

show
hide
Locator:
Data availability (Supplementary material)
Description:
-
Locator:
Electronic supplementary material (Supplementary material)
Description:
-

Creators

show
hide
 Creators:
Macuch Silva, Vinicius1, Author
Holler, Judith2, 3, 4, Author              
Ozyurek, Asli4, 5, 6, 7, Author              
Roberts, Seán G.8, Author
Affiliations:
1Institute of Cognitive Science, University of Osnabrück, Osnabrück, Germany, ou_persistent22              
2Communication in Social Interaction, Radboud University Nijmegen, External Organizations, ou_3055481              
3Language and Cognition Department, MPI for Psycholinguistics, Max Planck Society, Nijmegen, NL, ou_792548              
4Donders Institute for Brain, Cognition and Behaviour, External Organizations, ou_55236              
5Multimodal Language and Cognition, Radboud University Nijmegen, External Organizations, ou_3055480              
6Research Associates, MPI for Psycholinguistics, Max Planck Society, Wundtlaan 1, 6525 XD Nijmegen, NL, ou_2344700              
7Center for Language Studies, External Organizations, ou_55238              
8University of Bristol, Bristol, UK, ou_persistent22              

Content

show
hide
Free keywords: -
 Abstract: Face-to-face communication is multimodal at its core: it consists of a combination of vocal and visual signalling. However, current evidence suggests that, in the absence of an established communication system, visual signalling, especially in the form of visible gesture, is a more powerful form of communication than vocalisation, and therefore likely to have played a primary role in the emergence of human language. This argument is based on experimental evidence of how vocal and visual modalities (i.e., gesture) are employed to communicate about familiar concepts when participants cannot use their existing languages. To investigate this further, we introduce an experiment where pairs of participants performed a referential communication task in which they described unfamiliar stimuli in order to reduce reliance on conventional signals. Visual and auditory stimuli were described in three conditions: using visible gestures only, using non-linguistic vocalisations only and given the option to use both (multimodal communication). The results suggest that even in the absence of conventional signals, gesture is a more powerful mode of communication compared to vocalisation, but that there are also advantages to multimodality compared to using gesture alone. Participants with an option to produce multimodal signals had comparable accuracy to those using only gesture, but gained an efficiency advantage. The analysis of the interactions between participants showed that interactants developed novel communication systems for unfamiliar stimuli by deploying different modalities flexibly to suit their needs and by taking advantage of multimodality when required.

Details

show
hide
Language(s): eng - English
 Dates: 20192020-01-15
 Publication Status: Published online
 Pages: -
 Publishing info: -
 Table of Contents: -
 Rev. Method: Peer
 Identifiers: DOI: 10.1098/rsos.182056
 Degree: -

Event

show

Legal Case

show

Project information

show hide
Project name : INTERACT
Grant ID : 269484
Funding program : Funding Programme 7 (FP7)
Funding organization : European Commission (EC)

Source 1

show
hide
Title: Royal Society Open Science
Source Genre: Journal
 Creator(s):
Affiliations:
Publ. Info: London : Royal Society
Pages: - Volume / Issue: 7 Sequence Number: 182056 Start / End Page: - Identifier: ISSN: 2054-5703
CoNE: https://pure.mpg.de/cone/journals/resource/2054-5703