English
 
Help Privacy Policy Disclaimer
  Advanced SearchBrowse

Item

ITEM ACTIONSEXPORT
 
 
DownloadE-Mail
 Local TagsRelease HistoryDetailsSummary
  Using fuzzy string matching for automated assessment of listener transcripts in speech intelligibility studies

Bosker, H. R. (2021). Using fuzzy string matching for automated assessment of listener transcripts in speech intelligibility studies. Behavior Research Methods, 53(5), 1945-1953. doi:10.3758/s13428-021-01542-4.

Item is

 

show all hide all

Basic

show hide
Item Permalink: https://hdl.handle.net/21.11116/0000-0007-AAAC-8 Version Permalink: https://hdl.handle.net/21.11116/0000-0009-940E-1
Genre: Journal Article

Files

show Files
hide Files
:
Bosker2021_Article_UsingFuzzyStringMatchingForAut.pdf (Publisher version), 497KB
View   Save
File Permalink:
https://hdl.handle.net/21.11116/0000-0008-23AC-F
Name:
Bosker2021_Article_UsingFuzzyStringMatchingForAut.pdf
Description:
-
OA-Status:
Visibility:
Public
MIME-Type / Checksum:
application/pdf / [MD5]
Technical Metadata:
View
Copyright Date:
2021
Copyright Info:
This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
License:
http://creativecommons.org/licenses/by/4.0/
:
Bosker_2021_Using fuzzy string matching for automated....pdf (Publisher version), 509KB
View   Save
File Permalink:
https://hdl.handle.net/21.11116/0000-0009-940F-0
Name:
Bosker_2021_Using fuzzy string matching for automated....pdf
Description:
-
OA-Status:
Visibility:
Public
MIME-Type / Checksum:
application/pdf / [MD5]
Technical Metadata:
View
Copyright Date:
2021
Copyright Info:
This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
License:
http://creativecommons.org/licenses/by/4.0/

Locators

show

Creators

show
hide
 Creators:
Bosker, Hans R.1, 2, Author           
Affiliations:
1Psychology of Language Department, MPI for Psycholinguistics, Max Planck Society, ou_792545              
2Donders Institute for Brain, Cognition and Behaviour, External Organizations, ou_55236              

Content

show
hide
Free keywords: -
 Abstract: Many studies of speech perception assess the intelligibility of spoken sentence stimuli by means
of transcription tasks (‘type out what you hear’). The intelligibility of a given stimulus is then often
expressed in terms of percentage of words correctly reported from the target sentence. Yet scoring
the participants’ raw responses for words correctly identified from the target sentence is a time-
consuming task, and hence resource-intensive. Moreover, there is no consensus among speech
scientists about what specific protocol to use for the human scoring, limiting the reliability of
human scores. The present paper evaluates various forms of fuzzy string matching between
participants’ responses and target sentences, as automated metrics of listener transcript accuracy.
We demonstrate that one particular metric, the Token Sort Ratio, is a consistent, highly efficient,
and accurate metric for automated assessment of listener transcripts, as evidenced by high
correlations with human-generated scores (best correlation: r = 0.940) and a strong relationship to
acoustic markers of speech intelligibility. Thus, fuzzy string matching provides a practical tool for
assessment of listener transcript accuracy in large-scale speech intelligibility studies. See
https://tokensortratio.netlify.app for an online implementation.

Details

show
hide
Language(s): eng - English
 Dates: 2021-012021-03-102021-10
 Publication Status: Issued
 Pages: -
 Publishing info: -
 Table of Contents: -
 Rev. Type: Peer
 Identifiers: DOI: 10.3758/s13428-021-01542-4
 Degree: -

Event

show

Legal Case

show

Project information

show

Source 1

show
hide
Title: Behavior Research Methods
Source Genre: Journal
 Creator(s):
Affiliations:
Publ. Info: -
Pages: - Volume / Issue: 53 (5) Sequence Number: - Start / End Page: 1945 - 1953 Identifier: ISSN: 1554-3528
CoNE: https://pure.mpg.de/cone/journals/resource/1554-3528