Double sampling with multiple imputation to answer large sample meta-research questions: Introduction and illustration by evaluating adherence to two simple CONSORT guidelines
BACKGROUND: Meta-research can involve manual retrieval and evaluation of research, which is resource intensive. Creation of high throughput methods (e.g., search heuristics, crowdsourcing) has improved feasibility of large meta-research questions, but possibly at the cost of accuracy. OBJECTIVE: To...
Main Authors: | , , , |
---|---|
Format: | Article |
Language: | English |
Published: |
Frontiers Media S.A.
2015-03-01
|
Series: | Frontiers in Nutrition |
Subjects: | |
Online Access: | http://journal.frontiersin.org/Journal/10.3389/fnut.2015.00006/full |
id |
doaj-8d4802761f1743d1955d643053cf8ebb |
---|---|
record_format |
Article |
spelling |
doaj-8d4802761f1743d1955d643053cf8ebb2020-11-25T01:06:27ZengFrontiers Media S.A.Frontiers in Nutrition2296-861X2015-03-01210.3389/fnut.2015.00006123291Double sampling with multiple imputation to answer large sample meta-research questions: Introduction and illustration by evaluating adherence to two simple CONSORT guidelinesPatrice L. Capers0Andrew W Brown1John eDawson2John eDawson3David B Allison4David B Allison5David B Allison6David B Allison7University of Alabama at BirminghamUniversity of Alabama at BirminghamUniversity of Alabama at BirminghamUniversity of Alabama at BirminghamUniversity of Alabama at BirminghamUniversity of Alabama at BirminghamUniversity of Alabama at BirminghamUniversity of Alabama at BirminghamBACKGROUND: Meta-research can involve manual retrieval and evaluation of research, which is resource intensive. Creation of high throughput methods (e.g., search heuristics, crowdsourcing) has improved feasibility of large meta-research questions, but possibly at the cost of accuracy. OBJECTIVE: To evaluate the use of double sampling combined with multiple imputation (DS+MI) to address meta-research questions, using as an example adherence of PubMed entries to two simple Consolidated Standards of Reporting Trials (CONSORT) guidelines for titles and abstracts. METHODS: For the DS large sample, we retrieved all PubMed entries satisfying the filters: RCT; human; abstract available; and English language (n=322,107). For the DS subsample, we randomly sampled 500 entries from the large sample. The large sample was evaluated with a lower rigor, higher throughput (RLOTHI) method using search heuristics, while the subsample was evaluated using a higher rigor, lower throughput (RHITLO) human rating method. Multiple imputation of the missing-completely-at-random RHITLO data for the large sample was informed by: RHITLO data from the subsample; RLOTHI data from the large sample; whether a study was an RCT; and country and year of publication. RESULTS: The RHITLO and RLOTHI methods in the subsample largely agreed (phi coefficients: title=1.00, abstract=0.92). Compliance with abstract and title criteria has increased over time, with non-US countries improving more rapidly. DS+MI logistic regression estimates were more precise than subsample estimates (e.g., 95% CI for change in title and abstract compliance by Year: subsample RHITLO 1.050-1.174 vs. DS+MI 1.082-1.151). As evidence of improved accuracy, DS+MI coefficient estimates were closer to RHITLO than the large sample RLOTHI. CONCLUSIONS: Our results support our hypothesis that DS+MI would result in improved precision and accuracy. This method is flexible and may provide a practical way to examine large corpora of literature.http://journal.frontiersin.org/Journal/10.3389/fnut.2015.00006/fullmodelingadherenceCONSORTDouble samplingMultiple imputationMeta-research |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Patrice L. Capers Andrew W Brown John eDawson John eDawson David B Allison David B Allison David B Allison David B Allison |
spellingShingle |
Patrice L. Capers Andrew W Brown John eDawson John eDawson David B Allison David B Allison David B Allison David B Allison Double sampling with multiple imputation to answer large sample meta-research questions: Introduction and illustration by evaluating adherence to two simple CONSORT guidelines Frontiers in Nutrition modeling adherence CONSORT Double sampling Multiple imputation Meta-research |
author_facet |
Patrice L. Capers Andrew W Brown John eDawson John eDawson David B Allison David B Allison David B Allison David B Allison |
author_sort |
Patrice L. Capers |
title |
Double sampling with multiple imputation to answer large sample meta-research questions: Introduction and illustration by evaluating adherence to two simple CONSORT guidelines |
title_short |
Double sampling with multiple imputation to answer large sample meta-research questions: Introduction and illustration by evaluating adherence to two simple CONSORT guidelines |
title_full |
Double sampling with multiple imputation to answer large sample meta-research questions: Introduction and illustration by evaluating adherence to two simple CONSORT guidelines |
title_fullStr |
Double sampling with multiple imputation to answer large sample meta-research questions: Introduction and illustration by evaluating adherence to two simple CONSORT guidelines |
title_full_unstemmed |
Double sampling with multiple imputation to answer large sample meta-research questions: Introduction and illustration by evaluating adherence to two simple CONSORT guidelines |
title_sort |
double sampling with multiple imputation to answer large sample meta-research questions: introduction and illustration by evaluating adherence to two simple consort guidelines |
publisher |
Frontiers Media S.A. |
series |
Frontiers in Nutrition |
issn |
2296-861X |
publishDate |
2015-03-01 |
description |
BACKGROUND: Meta-research can involve manual retrieval and evaluation of research, which is resource intensive. Creation of high throughput methods (e.g., search heuristics, crowdsourcing) has improved feasibility of large meta-research questions, but possibly at the cost of accuracy. OBJECTIVE: To evaluate the use of double sampling combined with multiple imputation (DS+MI) to address meta-research questions, using as an example adherence of PubMed entries to two simple Consolidated Standards of Reporting Trials (CONSORT) guidelines for titles and abstracts. METHODS: For the DS large sample, we retrieved all PubMed entries satisfying the filters: RCT; human; abstract available; and English language (n=322,107). For the DS subsample, we randomly sampled 500 entries from the large sample. The large sample was evaluated with a lower rigor, higher throughput (RLOTHI) method using search heuristics, while the subsample was evaluated using a higher rigor, lower throughput (RHITLO) human rating method. Multiple imputation of the missing-completely-at-random RHITLO data for the large sample was informed by: RHITLO data from the subsample; RLOTHI data from the large sample; whether a study was an RCT; and country and year of publication. RESULTS: The RHITLO and RLOTHI methods in the subsample largely agreed (phi coefficients: title=1.00, abstract=0.92). Compliance with abstract and title criteria has increased over time, with non-US countries improving more rapidly. DS+MI logistic regression estimates were more precise than subsample estimates (e.g., 95% CI for change in title and abstract compliance by Year: subsample RHITLO 1.050-1.174 vs. DS+MI 1.082-1.151). As evidence of improved accuracy, DS+MI coefficient estimates were closer to RHITLO than the large sample RLOTHI. CONCLUSIONS: Our results support our hypothesis that DS+MI would result in improved precision and accuracy. This method is flexible and may provide a practical way to examine large corpora of literature. |
topic |
modeling adherence CONSORT Double sampling Multiple imputation Meta-research |
url |
http://journal.frontiersin.org/Journal/10.3389/fnut.2015.00006/full |
work_keys_str_mv |
AT patricelcapers doublesamplingwithmultipleimputationtoanswerlargesamplemetaresearchquestionsintroductionandillustrationbyevaluatingadherencetotwosimpleconsortguidelines AT andrewwbrown doublesamplingwithmultipleimputationtoanswerlargesamplemetaresearchquestionsintroductionandillustrationbyevaluatingadherencetotwosimpleconsortguidelines AT johnedawson doublesamplingwithmultipleimputationtoanswerlargesamplemetaresearchquestionsintroductionandillustrationbyevaluatingadherencetotwosimpleconsortguidelines AT johnedawson doublesamplingwithmultipleimputationtoanswerlargesamplemetaresearchquestionsintroductionandillustrationbyevaluatingadherencetotwosimpleconsortguidelines AT davidballison doublesamplingwithmultipleimputationtoanswerlargesamplemetaresearchquestionsintroductionandillustrationbyevaluatingadherencetotwosimpleconsortguidelines AT davidballison doublesamplingwithmultipleimputationtoanswerlargesamplemetaresearchquestionsintroductionandillustrationbyevaluatingadherencetotwosimpleconsortguidelines AT davidballison doublesamplingwithmultipleimputationtoanswerlargesamplemetaresearchquestionsintroductionandillustrationbyevaluatingadherencetotwosimpleconsortguidelines AT davidballison doublesamplingwithmultipleimputationtoanswerlargesamplemetaresearchquestionsintroductionandillustrationbyevaluatingadherencetotwosimpleconsortguidelines |
_version_ |
1725190112974733312 |