Bayesian l0-regularized least squares

• Main Authors: Polson, N.G (Author), Sun, L. (Author)
• Format: Article
• Language: English
• Published: John Wiley and Sons Ltd 2019
• Subjects: Bayesian variable selection; Computational efficiency; l0 regularization; L0- regularizations; proximal updating; Sampling; single best replacement; sparsity; spike-and-slab prior
• Online Access: View Fulltext in Publisher

 Bayesian l0-regularized least squares is a variable selection technique for high-dimensional predictors. The challenge is optimizing a nonconvex objective function via search over model space consisting of all possible predictor combinations. Spike-and-slab (aka Bernoulli-Gaussian) priors are the gold standard for Bayesian variable selection, with a caveat of computational speed and scalability. Single best replacement (SBR) provides a fast scalable alternative. We provide a link between Bayesian regularization and proximal updating, which provides an equivalence between finding a posterior mode and a posterior mean with a different regularization prior. This allows us to use SBR to find the spike-and-slab estimator. To illustrate our methodology, we provide simulation evidence and a real data example on the statistical properties and computational efficiency of SBR versus direct posterior sampling using spike-and-slab priors. Finally, we conclude with directions for future research. © 2018 John Wiley & Sons, Ltd.