On ridge regression and least absolute shrinkage and selection operator

• Main Author: AlNasser, Hassan
• Other Authors: Zhou, Julie
• Format: Others
• Language: English; en
• Published: 2017
• Subjects: LASSO; Ridge Regression; Bilevel Optimization
• Online Access: https://dspace.library.uvic.ca//handle/1828/8499

This thesis focuses on ridge regression (RR) and least absolute shrinkage and selection operator (lasso). Ridge properties are being investigated in great detail which include studying the bias, the variance and the mean squared error as a function of the tuning parameter. We also study the convexity of the trace of the mean squared error in terms of the tuning parameter. In addition, we examined some special properties of RR for factorial experiments. Not only do we review ridge properties, we also review lasso properties because they are somewhat similar. Rather than shrinking the estimates toward zero in RR, the lasso is able to provide a sparse solution, setting many coefficient estimates exaclty to zero. Furthermore, we try a new approach to solve the lasso problem by formulating it as a bilevel problem and implementing a new algorithm to solve this bilevel program. === Graduate