Bootstrapped Parameter Estimation in Ridge Regression with Multicollinearity and Multiple Outliers

An estimation of a regression is said to be superior if it is robust and resistant towards the presence of multicollinearity, outliers or non-normality. With regard to these problems, estimation using the OLS is inefficient since it could not produce the least variance. Disregarding this problem would potentially lead to severe statistical problems. Therefore, the main objective of the study is to employ a combination of a bootstrapping technique and robust methods in ridge regression model for data with simultaneous problems of multicollinearity and multiple outliers. This study employed the fixed - x resampling technique for robust ridge regression. The proposed method was expected to reduce the effect of the problems to the estimation results by producing lower standard error values as compared to the existing methods. The results of the study showed that the proposed technique was able to produce better parameter estimates with lower standard error values. A linear regression is used mainly to model a relationship between two or more than two variables. In general, the most popular method used for regression is Ordinary Least Squares (OLS) for its ease and simplicity. The OLS method is claimed to be unbiased, efficient and consistent estimator as compared to other linear unbiased estimators. However, this superiority features can be held only if the assumptions of classical linear regression model are satisfied. If the assumption is violated, the OLS method will no longer produce the least variance, leading to the inefficiency in estimating a model. One of the assumptions is that there is no exact linear relationship between the explanatory variables. Multicollinearity is said to present if this assumption is violated. In the presence of multicollinearity, the OLS estimator will produce infinite variance, which could lead to misleading interpretation of test statistics. If the purpose of a study is merely to predict the values of the dependent variable, multicollinearity is not a problem. However, the real issue arises if the study aims for parameter estimation. This is due to that, multicollinearity inflates the standard error values. A severe issue is concerned when the situation is affected by the presence of outliers. In general, outliers can be regarded as an observation that behaves differently as compared to the rest of observations. The presence of outliers will distort the parameter estimation of a model, hence inflate the statistical test and lead to misleading conclusions. Most of regression analysis assumes normality assumption of the error distribution. However in practice, there is a situation where the error distribution is not normal. In this case, disregarding the non normality of the error distribution will affect the estimation of a model. Ridge regression was firstly initiated by Hoerl (4) and Hoerl and Kennard (5) by introducing a biasing parameter with more stable and precise than the OLS estimator in handling the multicollinearity problem. Robust estimation is used mainly to handle the outliers problem by downweighting the effect of outliers. Robust estimator used in this study is a MM - estimator that was introduced by Yohai (10). The MM- estimator is an extended version of M-estimator that was previously proposed by Huber (6). This method is a combination of the properties of high asymptotic relative efficiency of M-estimator with the high breakdown of a class of S-estimator. Habshah and Marina (3) proposed a combination of ridge regression and robust techniques for multicollinearity and outliers problems. The study was further extended to