Abstract: We consider estimation of a linear or nonparametric additive model in which a few coefficients or additive components are ``large" and may be objects of substantive interest, whereas others are ``small" but not necessarily zero. The number of small coefficients or additive components may exceed the sample size. It is not known which coefficients or components are large and which are small. The large coefficients or additive components can be estimated with a smaller mean-square error or integrated mean-square error if the small ones can be identified and the covariates associated with them dropped from the model. We give conditions under which several penalized least squares procedures distinguish correctly between large and small coefficients or additive components with probability approaching 1 as the sample size increases. The results of Monte Carlo experiments and an empirical example illustrate the benefits of our methods.
Key words and phrases: High-dimensional data, penalized regression, variable selection.