Cross-sectional study of preprints and final journal publications from COVID-19 studies: discrepancies in results reporting and spin in interpretation
Objective To compare results reporting and the presence of spin in COVID-19 study preprints with their finalised journal publications.Design Cross-sectional study.Setting International medical literature.Participants Preprints and final journal publications of 67 interventional and observational stu...
Main Authors: | , , , , , , , , , , |
---|---|
Format: | Article |
Language: | English |
Published: |
BMJ Publishing Group
2021-07-01
|
Series: | BMJ Open |
Online Access: | https://bmjopen.bmj.com/content/11/7/e051821.full |
id |
doaj-880017df8ee94015bf88624745d9d3f7 |
---|---|
record_format |
Article |
spelling |
doaj-880017df8ee94015bf88624745d9d3f72021-08-07T16:32:53ZengBMJ Publishing GroupBMJ Open2044-60552021-07-0111710.1136/bmjopen-2021-051821Cross-sectional study of preprints and final journal publications from COVID-19 studies: discrepancies in results reporting and spin in interpretationRobin Featherstone0Lisa Bero1Matthew J Page2Jamie J Kirkham3Lisa Parker4Sally McDonald5Quinn Grundy6Kellia Chiu7Rosa Lawrence8Louis Leslie9Stephanie Boughton107 Editorial and Methods Department, Cochrane, London, UK1 General Internal Medicine/Public Health/Center for Bioethics and Humanities, University of Colorado—Anschutz Medical Campus, Denver, Colorado, USA4 School of Public Health and Preventive Medicine, Monash University, Melbourne, Victoria, Australia8 Biostatistics, Manchester University, Manchester, UK6 Charles Perkins Centre, The University of Sydney, Sydney, New South Wales, Australia3 Charles Perkins Centre and School of Pharmacy, Faculty of Medicine and Health, The University of Sydney, Sydney, New South Wales, Australia5 Faculty of Nursing, University of Sydney, Toronto, Ontario, Canada3 Charles Perkins Centre and School of Pharmacy, Faculty of Medicine and Health, The University of Sydney, Sydney, New South Wales, Australia2 Center for Bioethics and Humanities, University of Colorado - Anschutz Medical Center, Denver, Colorado, USA2 Center for Bioethics and Humanities, University of Colorado - Anschutz Medical Center, Denver, Colorado, USA7 Editorial and Methods Department, Cochrane, London, UKObjective To compare results reporting and the presence of spin in COVID-19 study preprints with their finalised journal publications.Design Cross-sectional study.Setting International medical literature.Participants Preprints and final journal publications of 67 interventional and observational studies of COVID-19 treatment or prevention from the Cochrane COVID-19 Study Register published between 1 March 2020 and 30 October 2020.Main outcome measures Study characteristics and discrepancies in (1) results reporting (number of outcomes, outcome descriptor, measure, metric, assessment time point, data reported, reported statistical significance of result, type of statistical analysis, subgroup analyses (if any), whether outcome was identified as primary or secondary) and (2) spin (reporting practices that distort the interpretation of results so they are viewed more favourably).Results Of 67 included studies, 23 (34%) had no discrepancies in results reporting between preprints and journal publications. Fifteen (22%) studies had at least one outcome that was included in the journal publication, but not the preprint; eight (12%) had at least one outcome that was reported in the preprint only. For outcomes that were reported in both preprints and journals, common discrepancies were differences in numerical values and statistical significance, additional statistical tests and subgroup analyses and longer follow-up times for outcome assessment in journal publications.At least one instance of spin occurred in both preprints and journals in 23/67 (34%) studies, the preprint only in 5 (7%), and the journal publications only in 2 (3%). Spin was removed between the preprint and journal publication in 5/67 (7%) studies; but added in 1/67 (1%) study.Conclusions The COVID-19 preprints and their subsequent journal publications were largely similar in reporting of study characteristics, outcomes and spin. All COVID-19 studies published as preprints and journal publications should be critically evaluated for discrepancies and spin.https://bmjopen.bmj.com/content/11/7/e051821.full |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Robin Featherstone Lisa Bero Matthew J Page Jamie J Kirkham Lisa Parker Sally McDonald Quinn Grundy Kellia Chiu Rosa Lawrence Louis Leslie Stephanie Boughton |
spellingShingle |
Robin Featherstone Lisa Bero Matthew J Page Jamie J Kirkham Lisa Parker Sally McDonald Quinn Grundy Kellia Chiu Rosa Lawrence Louis Leslie Stephanie Boughton Cross-sectional study of preprints and final journal publications from COVID-19 studies: discrepancies in results reporting and spin in interpretation BMJ Open |
author_facet |
Robin Featherstone Lisa Bero Matthew J Page Jamie J Kirkham Lisa Parker Sally McDonald Quinn Grundy Kellia Chiu Rosa Lawrence Louis Leslie Stephanie Boughton |
author_sort |
Robin Featherstone |
title |
Cross-sectional study of preprints and final journal publications from COVID-19 studies: discrepancies in results reporting and spin in interpretation |
title_short |
Cross-sectional study of preprints and final journal publications from COVID-19 studies: discrepancies in results reporting and spin in interpretation |
title_full |
Cross-sectional study of preprints and final journal publications from COVID-19 studies: discrepancies in results reporting and spin in interpretation |
title_fullStr |
Cross-sectional study of preprints and final journal publications from COVID-19 studies: discrepancies in results reporting and spin in interpretation |
title_full_unstemmed |
Cross-sectional study of preprints and final journal publications from COVID-19 studies: discrepancies in results reporting and spin in interpretation |
title_sort |
cross-sectional study of preprints and final journal publications from covid-19 studies: discrepancies in results reporting and spin in interpretation |
publisher |
BMJ Publishing Group |
series |
BMJ Open |
issn |
2044-6055 |
publishDate |
2021-07-01 |
description |
Objective To compare results reporting and the presence of spin in COVID-19 study preprints with their finalised journal publications.Design Cross-sectional study.Setting International medical literature.Participants Preprints and final journal publications of 67 interventional and observational studies of COVID-19 treatment or prevention from the Cochrane COVID-19 Study Register published between 1 March 2020 and 30 October 2020.Main outcome measures Study characteristics and discrepancies in (1) results reporting (number of outcomes, outcome descriptor, measure, metric, assessment time point, data reported, reported statistical significance of result, type of statistical analysis, subgroup analyses (if any), whether outcome was identified as primary or secondary) and (2) spin (reporting practices that distort the interpretation of results so they are viewed more favourably).Results Of 67 included studies, 23 (34%) had no discrepancies in results reporting between preprints and journal publications. Fifteen (22%) studies had at least one outcome that was included in the journal publication, but not the preprint; eight (12%) had at least one outcome that was reported in the preprint only. For outcomes that were reported in both preprints and journals, common discrepancies were differences in numerical values and statistical significance, additional statistical tests and subgroup analyses and longer follow-up times for outcome assessment in journal publications.At least one instance of spin occurred in both preprints and journals in 23/67 (34%) studies, the preprint only in 5 (7%), and the journal publications only in 2 (3%). Spin was removed between the preprint and journal publication in 5/67 (7%) studies; but added in 1/67 (1%) study.Conclusions The COVID-19 preprints and their subsequent journal publications were largely similar in reporting of study characteristics, outcomes and spin. All COVID-19 studies published as preprints and journal publications should be critically evaluated for discrepancies and spin. |
url |
https://bmjopen.bmj.com/content/11/7/e051821.full |
work_keys_str_mv |
AT robinfeatherstone crosssectionalstudyofpreprintsandfinaljournalpublicationsfromcovid19studiesdiscrepanciesinresultsreportingandspinininterpretation AT lisabero crosssectionalstudyofpreprintsandfinaljournalpublicationsfromcovid19studiesdiscrepanciesinresultsreportingandspinininterpretation AT matthewjpage crosssectionalstudyofpreprintsandfinaljournalpublicationsfromcovid19studiesdiscrepanciesinresultsreportingandspinininterpretation AT jamiejkirkham crosssectionalstudyofpreprintsandfinaljournalpublicationsfromcovid19studiesdiscrepanciesinresultsreportingandspinininterpretation AT lisaparker crosssectionalstudyofpreprintsandfinaljournalpublicationsfromcovid19studiesdiscrepanciesinresultsreportingandspinininterpretation AT sallymcdonald crosssectionalstudyofpreprintsandfinaljournalpublicationsfromcovid19studiesdiscrepanciesinresultsreportingandspinininterpretation AT quinngrundy crosssectionalstudyofpreprintsandfinaljournalpublicationsfromcovid19studiesdiscrepanciesinresultsreportingandspinininterpretation AT kelliachiu crosssectionalstudyofpreprintsandfinaljournalpublicationsfromcovid19studiesdiscrepanciesinresultsreportingandspinininterpretation AT rosalawrence crosssectionalstudyofpreprintsandfinaljournalpublicationsfromcovid19studiesdiscrepanciesinresultsreportingandspinininterpretation AT louisleslie crosssectionalstudyofpreprintsandfinaljournalpublicationsfromcovid19studiesdiscrepanciesinresultsreportingandspinininterpretation AT stephanieboughton crosssectionalstudyofpreprintsandfinaljournalpublicationsfromcovid19studiesdiscrepanciesinresultsreportingandspinininterpretation |
_version_ |
1721216962120384512 |