Predicting group-level outcome variables: An empirical comparison of analysis strategies

• Main Authors: Foster-Johnson, L. (Author), Kromrey, J.D (Author)
• Format: Article
• Language: English
• Published: Springer New York LLC 2018
• Subjects: Analysis of group means; analytic method; article; Bias; controlled study; Correlation of Data; effect size; error; Group-level outcomes; human; Humans; least square analysis; Least-Squares Analysis; Micro–macro data; multilevel analysis; Multilevel Analysis; observer variation; Observer Variation; outcome assessment; outcome variable; predictor variable; reliability; reproducibility; Reproducibility of Results; sample size; Sample Size; scientist; simulation; statistical bias
• Online Access: View Fulltext in Publisher

 This study provides a review of two methods for analyzing multilevel data with group-level outcome variables and compares them in a simulation study. The analytical methods included an unadjusted ordinary least squares (OLS) analysis of group means and a two-step adjustment of the group means suggested by Croon and van Veldhoven (2007). The Type I error control, power, bias, standard errors, and RMSE in parameter estimates were compared across design conditions that included manipulations of number of predictor variables, level of correlation between predictors, level of intraclass correlation, predictor reliability, effect size, and sample size. The results suggested that an OLS analysis of the group means, with White’s heteroscedasticity adjustment, provided more power for tests of group-level predictors, but less power for tests of individual-level predictors. Furthermore, this simple analysis avoided the extreme bias in parameter estimates and inadmissible solutions that were encountered with other strategies. These results were interpreted in terms of recommended analytical methods for applied researchers. © 2018, Psychonomic Society, Inc.