Improved two-stage model averaging for high-dimensional linear regression, with application to Riboflavin data analysis

• Main Author: Pan, J. (Author)
• Format: Article
• Language: English
• Published: BioMed Central Ltd 2021
• Subjects: Accurate prediction; Clustering algorithms; computer simulation; Computer Simulation; Corresponding weights; Cross-validation; data analysis; Data Analysis; Forecasting; High dimensional data; High-dimensional models; High-dimensional regression; High-dimensional regressions; Jackknife; Linear Models; Model averaging; Models, Statistical; Predictive performance; Quadratic programming; Regression analysis; riboflavin; Riboflavin; Squared prediction errors; statistical model; Variable selection; Variable selection methods
• Online Access: View Fulltext in Publisher

 Background: Model averaging has attracted increasing attention in recent years for the analysis of high-dimensional data. By weighting several competing statistical models suitably, model averaging attempts to achieve stable and improved prediction. In this paper, we develop a two-stage model averaging procedure to enhance accuracy and stability in prediction for high-dimensional linear regression. First we employ a high-dimensional variable selection method such as LASSO to screen redundant predictors and construct a class of candidate models, then we apply the jackknife cross-validation to optimize model weights for averaging. Results: In simulation studies, the proposed technique outperforms commonly used alternative methods under high-dimensional regression setting, in terms of minimizing the mean of the squared prediction error. We apply the proposed method to a riboflavin data, the result show that such method is quite efficient in forecasting the riboflavin production rate, when there are thousands of genes and only tens of subjects. Conclusions: Compared with a recent high-dimensional model averaging procedure (Ando and Li in J Am Stat Assoc 109:254–65, 2014), the proposed approach enjoys three appealing features thus has better predictive performance: (1) More suitable methods are applied for model constructing and weighting. (2) Computational flexibility is retained since each candidate model and its corresponding weight are determined in the low-dimensional setting and the quadratic programming is utilized in the cross-validation. (3) Model selection and averaging are combined in the procedure thus it makes full use of the strengths of both techniques. As a consequence, the proposed method can achieve stable and accurate predictions in high-dimensional linear models, and can greatly help practical researchers analyze genetic data in medical research. © 2021, The Author(s).