Ellipsoid Method for Linear Regression Parameters Determination

• Main Author: V. Stovba
• Format: Article
• Language: English
• Published: V.M. Glushkov Institute of Cybernetics 2020-10-01
• Series: Кібернетика та комп'ютерні технології
• Subjects: ellipsoid method; linear regression; outliers
• Online Access: http://cctech.org.ua/13-vertikalnoe-menyu-en/155-abstract-20-3-2-arte

Introduction. Linear regression parameters determination can be formulated as a non-smooth function minimization problem, which is Lp-norm of residual of the linear equations system. To solve it non-smooth function minimization methods can be used, e.g. subgradient methods. The article [7] considers ellipsoid method application for finding Lp-solution of redefined linear equations system with p in range [1,2]. The purpose of the paper is to extend the algorithm based on the ellipsoid method for a linear regression parameters determination problem with an arbitrary value of parameter p>=2 so that under big values of p the solution of the problem equals minimax method solution, which corresponds to p =infinity. To describe the formulation of observation approximation problem with quadratic function as linear regression parameters determination problem. To analyze algorithm work results for great number of observations and outliers. To compare the minimax method and the ellipsoid method algorithm work results for linear regression parameters determination problem with big values of parameter p. Results. The way of calculation of objective function and its subgradient values with large values of parameter p was developed and verified on example of observation approximation containing outliers with linear function. Algorithm based on ellipsoid method changes linear function parameters monotonically using parameter p adjusting, thereby permits to reject or consider these or those observations. It is shown in [3] that Least Absolute Deviations method (LAD) is advised to be used as far as it ignores outliers and reconstructs linear function accurately. Experiment results with big number of observations and outliers using p=1 confirmed that conclusion: LAD ignores outlier groups and approximates observations with linear function adequately. Least Square Method (LSM) deviates from optimal linear function if a group of outliers is present in particular area. In case of using big values of parameter p problem solution converges to minimax method solution. Conclusions. Algorithm based on ellipsoid method permits to determine linear regression parameters with arbitrary value of parameter p>=1. So, three known methods can be used – LAD, LSM and minimax method – as its special cases. Moreover, directing p to 1, intensity of outliers ignoring can be regulated, that gives a possibility to use external sources of information (expert opinions, measuring devices readings, statistical forecasts, etc.) for more correct and adequate approximation function reconstruction.