English
 
Help Privacy Policy Disclaimer
  Advanced SearchBrowse

Item

ITEM ACTIONSEXPORT

Released

Journal Article

Too Good to Be True: Bots and Bad Data From Mechanical Turk

MPS-Authors
/persons/resource/persons283161

Webb,  Margaret
Criminology, Max Planck Institute for the Study of Crime, Security and Law, Max Planck Society;

External Resource
Fulltext (restricted access)
There are currently no full texts shared for your IP range.
Fulltext (public)
There are no public fulltexts stored in PuRe
Supplementary Material (public)
There is no public supplementary material available
Citation

Webb, M., & Tangney, J. P. (2022). Too Good to Be True: Bots and Bad Data From Mechanical Turk. Perspectives on Psychological Science. doi:10.1177/17456916221120027.


Cite as: https://hdl.handle.net/21.11116/0000-000B-782B-F
Abstract
Psychology is moving increasingly toward digital sources of data, with Amazon’s Mechanical Turk (MTurk) at the forefront of that charge. In 2015, up to an estimated 45% of articles published in the top behavioral and social science journals included at least one study conducted on MTurk. In this article, I summarize my own experience with MTurk and how I deduced that my sample was—at best—only 2.6% valid, by my estimate. I share these results as a warning and call for caution. Recently, I conducted an online study via Amazon’s MTurk, eager and excited to collect my own data for the first time as a doctoral student. What resulted has prompted me to write this as a warning: it is indeed too good to be true. This is a summary of how I determined that, at best, I had gathered valid data from 14 human beings—2.6% of my participant sample (N = 529).