Back To Index Previous Article Next Article Full Text


Statistica Sinica 19 (2009), 449-471





BOOTSTRAP-BASED PENALTY CHOICE FOR THE LASSO,

ACHIEVING ORACLE PERFORMANCE


Peter Hall$^1$, Eun Ryung Lee$^2$ and Byeong U. Park$^2$


$^1$The University of Melbourne and $^2$Seoul National University


Abstract: In theory, if penalty parameters are chosen appropriately then the lasso can eliminate unnecessary variables in prediction problems, and improve the performance of predictors based on the variables that remain. However, standard methods for tuning-parameter choice, for example techniques based on the bootstrap or cross-validation, are not sufficiently accurate to achieve this level of precision. Until Zou's (2006) proposal for an inversely-weighted lasso, this difficulty led to speculation that it might not be possible to achieve oracle performance using the lasso. In the present paper we show that a straightforward application of the $m$-out-of-$n$ bootstrap produces adaptive penalty estimates that confer oracle properties on the lasso. The application is of interest in its own right since, unlike many uses of the $m$-out-of-$n$ bootstrap, it is not designed to estimate a non-normal distribution; the limiting distributions of regression parameter estimators are normal. Instead, the $m$-out-of-$n$ bootstrap overcomes the tendency of the standard bootstrap to confound the errors committed in determining whether or not a parameter value is zero, with estimation errors for nonzero parameters.



Key words and phrases: Adaptive inference, bootstrap, m-out-of-n bootstrap, optimality properties, prediction, regression, variable selection.

Back To Index Previous Article Next Article Full Text