hide
Free keywords:
-
Abstract:
Information in language is organized according to a
principle called information structure: new and important
information (focus) is highlighted and distinguished
from less important information (non-focus). Most
studies so far have been concerned with how focused
information is emphasized linguistically and suggest
that listeners expect focus to be accented and process it
more deeply than non-focus (Wang et al., 2011). Little is
known about how listeners deal with non-verbal cues
like beat gestures, which also emphasize the words
they accompany, similarly to pitch accent. ERP studies
suggest that beat gestures facilitate the processing of
phonological, syntactic, and semantic aspects of speech
(Biau, & Soto-Faraco, 2013; Holle et al., 2012; Wang &
Chu, 2013). It is unclear whether listeners expect beat
gestures to be aligned with the information structure
of the message. The present ERP study addresses this
question by testing whether beat gestures modulate
the processing of accented-focused vs. unaccented-non
focused words in context in a similar way. Participantswatched movies with short dialogues and performed a
comprehension task. In each dialogue, the answer “He
bought the books via amazon” contained a target word
(“books”) which was combined with a beat gesture, a
control hand movement (e.g., self touching movement)
or no gesture. Based on the preceding context, the target
word was either in focus and accented, when preceded
by a question like “Did the student buy the books or
the magazines via Amazon?”, or the target word was in
non-focus and unaccented, when preceded by a question
like “Did the student buy the books via Amazon or via
Marktplaats?”. The gestures started 500 ms prior to
the target word. All gesture parameters (hand shape,
naturalness, emphasis, duration, and gesture-speech
alignment) were determined in behavioural tests. ERPs
were time-locked to gesture onset to examine gesture
effects, and to target word onset for pitch accent effects.
We applied a cluster-based random permutation analysis
to test for main effects and gesture-accent interactions in
both time-locking procedures. We found that accented
words elicited a positive main effect between 300-600
ms post target onset. Words accompanied by a beat
gesture and a control movement elicited sustained
positivities between 200-1300 ms post gesture onset.
These independent effects of pitch accent and beat
gesture are in line with previous findings (Dimitrova
et al., 2012; Wang & Chu, 2013). We also found an
interaction between control gesture and pitch accent
(1200-1300 ms post gesture onset), showing that accented
words accompanied by a control movement elicited a
negativity relative to unaccented words. The present data
show that beat gestures do not differentially modulate
the processing of accented-focused vs. unaccented-non
focused words. Beat gestures engage a positive and long
lasting neural signature, which appears independent
from the information structure of the message. Our study
suggests that non-verbal cues like beat gestures play a
unique role in emphasizing information in speech.
