Abstract: Gene-environment (G✕E) interactions play a crucial role in many complex diseases. Many studies have highlighted the importance of the linear and nonlinear effects of G✕E interactions to the risk of contracting diseases. Linear effects can be modeled parametrically, whereas nonlinear effects are typically modeled and estimated using nonparametric functions under the framework of partial linear models. Because of the difference in the rates of convergence of the parametric and nonparametric parts, few statistical studies have assessed the simultaneous effects of the linear and nonlinear effects of G✕E interactions in the context of a partial linear model. In this study, we consider a hypothesis test to simultaneously detect the linear and nonlinear effects in a generalized partial linear varying-coefficient model. We propose a B-spline backfitted kernel method to estimate the effect of nonlinear interactions. A Wald-type statistic is constructed for the joint testing problem based on the nonparametric generalized likelihood ratio statistic. We show that the joint test statistic asymptotically follows a χ²-distribution under the null hypothesis of no G✕E interaction effect, and a noncentral χ²-distribution under the null hypothesis of no G✕E interaction effect-distribution under the alternative. Moreover, the proposed test can simultaneously detect alternatives at optimal rates for both the parametric and the nonparametric components. The utility of the method is demonstrated using extensive simulations and a case study.

Key words and phrases: B-spline back-fitting, genetic association, non-linear G✕E interaction, partial linear effect.