Abstract: In classical smoothing splines, the smoothness is controlled by a single smoothing parameter that penalizes the roughness uniformly across the whole domain. Adaptive smoothing splines extend this framework to allow the smoothing parameter to change in the domain, adapting to the change of roughness. In this article we propose a data driven method to nonparametrically model the penalty function. We propose to approximate the penalty function by a step function whose segmentation is data driven, and to estimate it by maximizing the generalized likelihood. A complexity penalty is added to the generalized likelihood in selecting the best step function from a collection of candidates. A state space representation for the adaptive smoothing splines is derived to ease the computational demand. To allow for fast search among the candidate models, we impose a binary tree structure on the penalty function and propose an efficient search algorithm. We show the consistency of the final estimate. We demonstrate the effectiveness of the method through simulations and a data example.
Key words and phrases: Binary tree, complexity penalty, generalized maximum likelihood, model selection, state space method.