Variable selection using P‐splines

Selecting among a large set of variables those that influence most a response variable is an important problem in statistics. When the assumed regression model involves a nonparametric component, penalized regression techniques, and in particular P‐splines, are among the commonly used methods. The aim of this paper is to provide a brief review of variable selection methods using P‐splines. Starting from multiple linear regression models, with least‐squares regression, and Ridge regression, we review standard methods that perform variable selection, such as LASSO, nonnegative garrote, the SCAD method, etc. We briefly discuss a general framework of penalization and regularization methods. Going toward more flexible regression models, with some nonparametric component(s), we discuss P‐splines estimation. For some examples of flexible regression models, we then review a few variable selection methods using P‐splines. A brief discussion on grouped regularization techniques and on a robust variable selection method is given. Furthermore, we mention key ingredients in Bayesian approaches, and end the paper by drawing the attention to several other issues in variable selection with P‐splines. Throughout the paper we provide some illustrations.WIREs Comput Stat2015, 7:1–20. doi: 10.1002/wics.1327 This article is categorized under: Statistical and Graphical Methods of Data Analysis > Nonparametric Methods Statistical Models > Model Selection