Systems Perspective of Amazon Mechanical Turk for Organizational Research: Review and Recommendations
Amazon Mechanical Turk (MTurk) is becoming a prevalent source of quick and cost effective data for organizational research, but there are questions about the appropriateness of the platform for organizational research. To answer these questions, we conducted an integrative review based on 75 papers...
Main Authors: | , , |
---|---|
Format: | Article |
Language: | English |
Published: |
Frontiers Media S.A.
2017-08-01
|
Series: | Frontiers in Psychology |
Subjects: | |
Online Access: | http://journal.frontiersin.org/article/10.3389/fpsyg.2017.01359/full |
id |
doaj-a13f953f42db471aa4e7eb2fcb8eaadf |
---|---|
record_format |
Article |
spelling |
doaj-a13f953f42db471aa4e7eb2fcb8eaadf2020-11-24T21:17:47ZengFrontiers Media S.A.Frontiers in Psychology1664-10782017-08-01810.3389/fpsyg.2017.01359274407Systems Perspective of Amazon Mechanical Turk for Organizational Research: Review and RecommendationsMelissa G. Keith0Louis Tay1Peter D. Harms2Department of Psychological Sciences, Purdue UniversityWest Lafayette, IN, United StatesDepartment of Psychological Sciences, Purdue UniversityWest Lafayette, IN, United StatesDepartment of Management, The University of AlabamaTuscaloosa, AL, United StatesAmazon Mechanical Turk (MTurk) is becoming a prevalent source of quick and cost effective data for organizational research, but there are questions about the appropriateness of the platform for organizational research. To answer these questions, we conducted an integrative review based on 75 papers evaluating the MTurk platform and 250 MTurk samples used in organizational research. This integrative review provides four contributions: (1) we analyze the trends associated with the use of MTurk samples in organizational research; (2) we develop a systems perspective (recruitment system, selection system, and work management system) to synthesize and organize the key factors influencing data collected on MTurk that may affect generalizability and data quality; (3) within each factor, we also use available MTurk samples from the organizational literature to analyze key issues (e.g., sample characteristics, use of attention checks, payment); and (4) based on our review, we provide specific recommendations and a checklist for data reporting in order to improve data transparency and enable further research on this issue.http://journal.frontiersin.org/article/10.3389/fpsyg.2017.01359/fullmechanical turksamplingreviewmeta-analysisresearch design |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Melissa G. Keith Louis Tay Peter D. Harms |
spellingShingle |
Melissa G. Keith Louis Tay Peter D. Harms Systems Perspective of Amazon Mechanical Turk for Organizational Research: Review and Recommendations Frontiers in Psychology mechanical turk sampling review meta-analysis research design |
author_facet |
Melissa G. Keith Louis Tay Peter D. Harms |
author_sort |
Melissa G. Keith |
title |
Systems Perspective of Amazon Mechanical Turk for Organizational Research: Review and Recommendations |
title_short |
Systems Perspective of Amazon Mechanical Turk for Organizational Research: Review and Recommendations |
title_full |
Systems Perspective of Amazon Mechanical Turk for Organizational Research: Review and Recommendations |
title_fullStr |
Systems Perspective of Amazon Mechanical Turk for Organizational Research: Review and Recommendations |
title_full_unstemmed |
Systems Perspective of Amazon Mechanical Turk for Organizational Research: Review and Recommendations |
title_sort |
systems perspective of amazon mechanical turk for organizational research: review and recommendations |
publisher |
Frontiers Media S.A. |
series |
Frontiers in Psychology |
issn |
1664-1078 |
publishDate |
2017-08-01 |
description |
Amazon Mechanical Turk (MTurk) is becoming a prevalent source of quick and cost effective data for organizational research, but there are questions about the appropriateness of the platform for organizational research. To answer these questions, we conducted an integrative review based on 75 papers evaluating the MTurk platform and 250 MTurk samples used in organizational research. This integrative review provides four contributions: (1) we analyze the trends associated with the use of MTurk samples in organizational research; (2) we develop a systems perspective (recruitment system, selection system, and work management system) to synthesize and organize the key factors influencing data collected on MTurk that may affect generalizability and data quality; (3) within each factor, we also use available MTurk samples from the organizational literature to analyze key issues (e.g., sample characteristics, use of attention checks, payment); and (4) based on our review, we provide specific recommendations and a checklist for data reporting in order to improve data transparency and enable further research on this issue. |
topic |
mechanical turk sampling review meta-analysis research design |
url |
http://journal.frontiersin.org/article/10.3389/fpsyg.2017.01359/full |
work_keys_str_mv |
AT melissagkeith systemsperspectiveofamazonmechanicalturkfororganizationalresearchreviewandrecommendations AT louistay systemsperspectiveofamazonmechanicalturkfororganizationalresearchreviewandrecommendations AT peterdharms systemsperspectiveofamazonmechanicalturkfororganizationalresearchreviewandrecommendations |
_version_ |
1726012127378931712 |