Analytic and Bootstrap Confidence Intervals for the Common-Language Effect Size Estimate

• Main Authors: Johnson Ching-Hong Li, Virginia Man Chung Tze
• Format: Article
• Language: English
• Published: PsychOpen 2021-03-01
• Series: Methodology
• Subjects: common-language effect size; confidence intervals; bootstrapping; monte carlo simulation; probability-of-superiority
• Online Access: https://meth.psychopen.eu/index.php/meth/article/view/4495
• ISSN: 1614-2241

Evaluating how an effect-size estimate performs between two continuous variables based on the common-language effect size (CLES) has received increasing attention. While Blomqvist (1950; https://doi.org/10.1214/aoms/1177729754) developed a parametric estimator (q') for the CLES, there has been limited progress in further refining CLES. This study: a) extends Blomqvist’s work by providing a mathematical foundation for Bp (a non-parametric version of CLES) and an analytic approach for estimating its standard error; and b) evaluates the performance of the analytic and bootstrap confidence intervals (CIs) for Bp. The simulation shows that the bootstrap bias-corrected-and-accelerated interval (BCaI) has the best protected Type 1 error rate with a slight compromise in Power, whereas the analytic-t CI has the highest overall Power but with a Type 1 error slightly larger than the nominal value. This study also uses a real-world data-set to demonstrate the applicability of the CLES in measuring the relationship between age and sexual compulsivity.