Abstract: We propose a new penalized likelihood method for model selection and nonparametric regression in exponential families. In the framework of smoothing spline ANOVA, our method employs a regularization with the penalty functional being the sum of the reproducing kernel Hilbert space norms of functional components in the ANOVA decomposition. It generalizes the LASSO in the linear regression to the nonparametric context, and conducts component selection and smoothing simultaneously. Continuous and categorical variables are treated in a unified fashion. We discuss the connection of the method to the traditional smoothing spline penalized likelihood estimation. We show that an equivalent formulation of the method leads naturally to an iterative algorithm. Simulations and examples are used to demonstrate the performances of the method.
Key words and phrases: Exponential family, LASSO, nonparametric regression, penalized likelihood, smoothing spline ANOVA.