Abstract: The time-varying coefficient Cox model has been widely studied and popularly used in survival data analysis due to its flexibility for modeling covariate effects. It is of great practical interest to accurately identify the structure of covariate effects in a time-varying coefficient Cox model, covariates with null effect, constant effect and truly time-varying effect, and estimate the corresponding regression coefficients. Combining the ideas of local polynomial smoothing and group nonnegative garrote, we develop a new penalization approach to achieve such goals. Our method is able to identify the underlying true model structure with probability tending to one and can simultaneously estimate the time-varying coefficients consistently. The asymptotic normalities of the resulting estimators are established. We demonstrate the performance of our method using simulations and an application to the primary biliary cirrhosis data.

Key words and phrases: Group nonnegative garrote, local polynomial smoothing, model selection, time-varying coefficient Cox model.