Overestimation of Action-Game Training Effects: Publication Bias and Salami Slicing
Does playing action video games improve performance on tests of cognitive ability? A recent meta-analysis (Bediou et al., 2018a) summarized the available evidence and concluded that it can. Their analysis, however, did not adequately correct for publication bias. We re-analyzed the same set of studi...
Main Authors: | , , , |
---|---|
Format: | Article |
Language: | English |
Published: |
University of California Press
2019-07-01
|
Series: | Collabra: Psychology |
Subjects: | |
Online Access: | https://www.collabra.org/articles/231 |
id |
doaj-883998e0285146c2bdaaa6315879c0fc |
---|---|
record_format |
Article |
spelling |
doaj-883998e0285146c2bdaaa6315879c0fc2020-11-24T21:56:05ZengUniversity of California PressCollabra: Psychology2474-73942019-07-015110.1525/collabra.231138Overestimation of Action-Game Training Effects: Publication Bias and Salami SlicingJoseph Hilgard0Giovanni Sala1Walter R. Boot2Daniel J. Simons3Illinois State University, Normal, IllinoisOsaka University, OsakaFlorida State University, Tallahassee, FloridaUniversity of Illinois Urbana-ChampaignDoes playing action video games improve performance on tests of cognitive ability? A recent meta-analysis (Bediou et al., 2018a) summarized the available evidence and concluded that it can. Their analysis, however, did not adequately correct for publication bias. We re-analyzed the same set of studies with more appropriate adjustments for publication bias and found minimal evidence for transfer of training to cognitive ability measures. Instead, it is possible that there are little or no benefits, just publication bias — the exclusion of non-significant results from the published literature. That bias may be the cause of a lab effect reported in the original meta-analysis. The meta-analysis showed that studies from the Bavelier lab (the senior author of the meta-analysis) reported larger effects than other labs. We show that many of these original studies distributed different outcomes from the same or highly overlapping sets of participants across publications without noting the overlap. This salami-slicing might contribute to the extent of publication bias in the literature. More compelling, independent, and transparent evidence is needed before concluding that action video game training transfers to performance on other cognitive tasks.https://www.collabra.org/articles/231video game trainingaction game trainingcognitive trainingmeta-analysispublication biassalami slicing |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Joseph Hilgard Giovanni Sala Walter R. Boot Daniel J. Simons |
spellingShingle |
Joseph Hilgard Giovanni Sala Walter R. Boot Daniel J. Simons Overestimation of Action-Game Training Effects: Publication Bias and Salami Slicing Collabra: Psychology video game training action game training cognitive training meta-analysis publication bias salami slicing |
author_facet |
Joseph Hilgard Giovanni Sala Walter R. Boot Daniel J. Simons |
author_sort |
Joseph Hilgard |
title |
Overestimation of Action-Game Training Effects: Publication Bias and Salami Slicing |
title_short |
Overestimation of Action-Game Training Effects: Publication Bias and Salami Slicing |
title_full |
Overestimation of Action-Game Training Effects: Publication Bias and Salami Slicing |
title_fullStr |
Overestimation of Action-Game Training Effects: Publication Bias and Salami Slicing |
title_full_unstemmed |
Overestimation of Action-Game Training Effects: Publication Bias and Salami Slicing |
title_sort |
overestimation of action-game training effects: publication bias and salami slicing |
publisher |
University of California Press |
series |
Collabra: Psychology |
issn |
2474-7394 |
publishDate |
2019-07-01 |
description |
Does playing action video games improve performance on tests of cognitive ability? A recent meta-analysis (Bediou et al., 2018a) summarized the available evidence and concluded that it can. Their analysis, however, did not adequately correct for publication bias. We re-analyzed the same set of studies with more appropriate adjustments for publication bias and found minimal evidence for transfer of training to cognitive ability measures. Instead, it is possible that there are little or no benefits, just publication bias — the exclusion of non-significant results from the published literature. That bias may be the cause of a lab effect reported in the original meta-analysis. The meta-analysis showed that studies from the Bavelier lab (the senior author of the meta-analysis) reported larger effects than other labs. We show that many of these original studies distributed different outcomes from the same or highly overlapping sets of participants across publications without noting the overlap. This salami-slicing might contribute to the extent of publication bias in the literature. More compelling, independent, and transparent evidence is needed before concluding that action video game training transfers to performance on other cognitive tasks. |
topic |
video game training action game training cognitive training meta-analysis publication bias salami slicing |
url |
https://www.collabra.org/articles/231 |
work_keys_str_mv |
AT josephhilgard overestimationofactiongametrainingeffectspublicationbiasandsalamislicing AT giovannisala overestimationofactiongametrainingeffectspublicationbiasandsalamislicing AT walterrboot overestimationofactiongametrainingeffectspublicationbiasandsalamislicing AT danieljsimons overestimationofactiongametrainingeffectspublicationbiasandsalamislicing |
_version_ |
1725859520316440576 |