Multiple Regression and Model Building

Multiple regression, where more than one predictor variable is used to estimate a response variable, is introduced by way of an example. To allow for inference, the multiple regression model is defined, with both model and inferential methods representing extensions of the simple linear regression case. Next, regression with categorical predictors (indicator variables) is explained. The problems of multicollinearity are examined; multicollinearity represents an unstable response surface due to overly correlated predictors. The variance inflation factor is defined, as an aid in identifying multicollinear predictors. Variable selection methods are then provided, including forward selection, backward elimination, stepwise, and best-subsets regression. Mallows'Cp statistic is defined, as an aid in variable selection. Finally, methods for using the principal components as predictors in multiple regression are discussed.