Abstract: We consider a functional regression model in the framework of reproducing kernel Hilbert spaces, where the interaction effect of two functional predictors, as well as their main effects, over the functional response is of interest. The regression component of our model is expressed by one trivariate coefficient function, the functional ANOVA decomposition of which yields the main and interaction effects. The trivariate coefficient function is estimated by optimizing a penalized least squares objective with a roughness penalty on the function estimate. The estimation procedure can be implemented easily using standard numerical tools. Asymptotic results for the proposed model, with or without functional measurement errors, are established under the reproducing kernel Hilbert space (RKHS) framework. Extensive numerical studies show the advantages of the proposed method over existing methods in terms of the prediction and estimation of the coefficient functions. An application to the histone modifications and gene expressions of a liver cancer cell line further demonstrates the better prediction accuracy of the proposed method over that of its competitors.

Key words and phrases: Functional ANOVA, functional interaction, function-on-function regression, measurement errors, minimax convergence rate, penalized least squares, tensor product.