hide
Free keywords:
memory, piano, eye-tracking, dynamic time warping, matrix profile, motif, audio indexing
Abstract:
While the eyes serve an obvious function in the context of music reading, their role during memorized music
performance (i.e., when there is no score) is currently unknown. Given previous work showing relationships
between eye movements and body movements and eye movements and memory retrieval, here I ask 1) whether
eye movements become a stable aspect of the memorized music (motor) performance, and 2) whether the structure
of the music is reflected in eye movement patterns. In this case study, three pianists chose two pieces to play from
memory. They came into the lab on four different days, separated by at least 12hrs, and played their two pieces
three times each. To answer 1), I compared dynamic time warping cost within vs. between pieces, and found
significantly lower warping costs within piece, for both horizontal and vertical eye movement time series,
providing a first proof-of-concept that eye movement patterns are conserved across repeated memorized music
performances. To answer 2), I used the Matrix Profiles of the eye movement time series to automatically detect
motifs (repeated patterns). By then analyzing participants’ recorded audio at moments of detected ocular motifs,
repeated sections of music could be identified (confirmed auditorily and with inspection of the extracted pitch and
amplitude envelopes of the indexed audio snippets). Overall, the current methods provide a promising approach
for future studies of music performance, enabling exploration of the relationship between body movements, eye
movements, and musical processing.
