Three Statistical Approaches for Assessment of Intervention Effects: A Primer for Practitioners
Lihua Li,1– 3 Meaghan S Cuerden,4 Bian Liu,1,3,5 Salimah Shariff,6 Arsh K Jain,4,6 Madhu Mazumdar1– 3 1Institute for Healthcare Delivery Science, Icahn School of Medicine at Mount Sinai, New York, NY, USA; 2Department of Population Health Science and Policy, Icahn School of Medic...
Main Authors: | , , , , , |
---|---|
Format: | Article |
Language: | English |
Published: |
Dove Medical Press
2021-02-01
|
Series: | Risk Management and Healthcare Policy |
Subjects: | |
Online Access: | https://www.dovepress.com/three-statistical-approaches-for-assessment-of-intervention-effects-a--peer-reviewed-article-RMHP |
id |
doaj-2e7e552799eb45979bb48e82fe51c714 |
---|---|
record_format |
Article |
spelling |
doaj-2e7e552799eb45979bb48e82fe51c7142021-02-23T20:04:45ZengDove Medical PressRisk Management and Healthcare Policy1179-15942021-02-01Volume 1475777062362Three Statistical Approaches for Assessment of Intervention Effects: A Primer for PractitionersLi LCuerden MSLiu BShariff SJain AKMazumdar MLihua Li,1– 3 Meaghan S Cuerden,4 Bian Liu,1,3,5 Salimah Shariff,6 Arsh K Jain,4,6 Madhu Mazumdar1– 3 1Institute for Healthcare Delivery Science, Icahn School of Medicine at Mount Sinai, New York, NY, USA; 2Department of Population Health Science and Policy, Icahn School of Medicine at Mount Sinai, New York, NY, USA; 3Tisch Cancer Institute, Icahn School of Medicine at Mount Sinai, New York, NY, USA; 4London Health Sciences Centre, London, Ontario, Canada; 5Institute for Translational Epidemiology, Icahn School of Medicine at Mount Sinai, New York, NY, USA; 6Institute for Clinical Evaluative Sciences, Toronto, Ontario, CanadaCorrespondence: Madhu MazumdarDepartment of Population Health Science and Policy, Icahn School of Medicine at Mount Sinai, Mount Sinai Hospital, 1425 Madison Avenue, New York, NY, USATel +1 212-659-1470Email Madhu.Mazumdar@mountsinai.orgIntroduction: Statistical methods to assess the impact of an intervention are increasingly used in clinical research settings. However, a comprehensive review of the methods geared toward practitioners is not yet available.Methods and Materials: We provide a comprehensive review of three methods to assess the impact of an intervention: difference-in-differences (DID), segmented regression of interrupted time series (ITS), and interventional autoregressive integrated moving average (ARIMA). We also compare the methods, and provide illustration of their use through three important healthcare-related applications.Results: In the first example, the DID estimate of the difference in health insurance coverage rates between expanded states and unexpanded states in the post-Medicaid expansion period compared to the pre-expansion period was 5.93 (95% CI, 3.99 to 7.89) percentage points. In the second example, a comparative segmented regression of ITS analysis showed that the mean imaging order appropriateness score in the emergency department at a tertiary care hospital exceeded that of the inpatient setting with a level change difference of 0.63 (95% CI, 0.53 to 0.73) and a trend change difference of 0.02 (95% CI, 0.01 to 0.03) after the introduction of a clinical decision support tool. In the third example, the results from an interventional ARIMA analysis show that numbers of creatinine clearance tests decreased significantly within months of the start of eGFR reporting, with a magnitude of drop equal to − 0.93 (95% CI, − 1.22 to − 0.64) tests per 100,000 adults and a rate of drop equal to 0.97 (95% CI, 0.95 to 0.99) tests per 100,000 per adults per month.Discussion: When choosing the appropriate method to model the intervention effect, it is necessary to consider the structure of the data, the study design, availability of an appropriate comparison group, sample size requirements, whether other interventions occur during the study window, and patterns in the data.Keywords: difference-in-difference, interrupted time series, segmented regression, autoregressive integrated moving averagehttps://www.dovepress.com/three-statistical-approaches-for-assessment-of-intervention-effects-a--peer-reviewed-article-RMHPdifference-in-differenceinterrupted time seriessegmented regressionautoregressive intergrated moving average |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Li L Cuerden MS Liu B Shariff S Jain AK Mazumdar M |
spellingShingle |
Li L Cuerden MS Liu B Shariff S Jain AK Mazumdar M Three Statistical Approaches for Assessment of Intervention Effects: A Primer for Practitioners Risk Management and Healthcare Policy difference-in-difference interrupted time series segmented regression autoregressive intergrated moving average |
author_facet |
Li L Cuerden MS Liu B Shariff S Jain AK Mazumdar M |
author_sort |
Li L |
title |
Three Statistical Approaches for Assessment of Intervention Effects: A Primer for Practitioners |
title_short |
Three Statistical Approaches for Assessment of Intervention Effects: A Primer for Practitioners |
title_full |
Three Statistical Approaches for Assessment of Intervention Effects: A Primer for Practitioners |
title_fullStr |
Three Statistical Approaches for Assessment of Intervention Effects: A Primer for Practitioners |
title_full_unstemmed |
Three Statistical Approaches for Assessment of Intervention Effects: A Primer for Practitioners |
title_sort |
three statistical approaches for assessment of intervention effects: a primer for practitioners |
publisher |
Dove Medical Press |
series |
Risk Management and Healthcare Policy |
issn |
1179-1594 |
publishDate |
2021-02-01 |
description |
Lihua Li,1– 3 Meaghan S Cuerden,4 Bian Liu,1,3,5 Salimah Shariff,6 Arsh K Jain,4,6 Madhu Mazumdar1– 3 1Institute for Healthcare Delivery Science, Icahn School of Medicine at Mount Sinai, New York, NY, USA; 2Department of Population Health Science and Policy, Icahn School of Medicine at Mount Sinai, New York, NY, USA; 3Tisch Cancer Institute, Icahn School of Medicine at Mount Sinai, New York, NY, USA; 4London Health Sciences Centre, London, Ontario, Canada; 5Institute for Translational Epidemiology, Icahn School of Medicine at Mount Sinai, New York, NY, USA; 6Institute for Clinical Evaluative Sciences, Toronto, Ontario, CanadaCorrespondence: Madhu MazumdarDepartment of Population Health Science and Policy, Icahn School of Medicine at Mount Sinai, Mount Sinai Hospital, 1425 Madison Avenue, New York, NY, USATel +1 212-659-1470Email Madhu.Mazumdar@mountsinai.orgIntroduction: Statistical methods to assess the impact of an intervention are increasingly used in clinical research settings. However, a comprehensive review of the methods geared toward practitioners is not yet available.Methods and Materials: We provide a comprehensive review of three methods to assess the impact of an intervention: difference-in-differences (DID), segmented regression of interrupted time series (ITS), and interventional autoregressive integrated moving average (ARIMA). We also compare the methods, and provide illustration of their use through three important healthcare-related applications.Results: In the first example, the DID estimate of the difference in health insurance coverage rates between expanded states and unexpanded states in the post-Medicaid expansion period compared to the pre-expansion period was 5.93 (95% CI, 3.99 to 7.89) percentage points. In the second example, a comparative segmented regression of ITS analysis showed that the mean imaging order appropriateness score in the emergency department at a tertiary care hospital exceeded that of the inpatient setting with a level change difference of 0.63 (95% CI, 0.53 to 0.73) and a trend change difference of 0.02 (95% CI, 0.01 to 0.03) after the introduction of a clinical decision support tool. In the third example, the results from an interventional ARIMA analysis show that numbers of creatinine clearance tests decreased significantly within months of the start of eGFR reporting, with a magnitude of drop equal to − 0.93 (95% CI, − 1.22 to − 0.64) tests per 100,000 adults and a rate of drop equal to 0.97 (95% CI, 0.95 to 0.99) tests per 100,000 per adults per month.Discussion: When choosing the appropriate method to model the intervention effect, it is necessary to consider the structure of the data, the study design, availability of an appropriate comparison group, sample size requirements, whether other interventions occur during the study window, and patterns in the data.Keywords: difference-in-difference, interrupted time series, segmented regression, autoregressive integrated moving average |
topic |
difference-in-difference interrupted time series segmented regression autoregressive intergrated moving average |
url |
https://www.dovepress.com/three-statistical-approaches-for-assessment-of-intervention-effects-a--peer-reviewed-article-RMHP |
work_keys_str_mv |
AT lil threestatisticalapproachesforassessmentofinterventioneffectsaprimerforpractitioners AT cuerdenms threestatisticalapproachesforassessmentofinterventioneffectsaprimerforpractitioners AT liub threestatisticalapproachesforassessmentofinterventioneffectsaprimerforpractitioners AT shariffs threestatisticalapproachesforassessmentofinterventioneffectsaprimerforpractitioners AT jainak threestatisticalapproachesforassessmentofinterventioneffectsaprimerforpractitioners AT mazumdarm threestatisticalapproachesforassessmentofinterventioneffectsaprimerforpractitioners |
_version_ |
1724254160889577472 |