Statistica Sinica 17(2007), 481-504

EMPIRICAL BAYES DENSITY REGRESSION

David B. Dunson

National Institute of Environmental Health Sciences

Abstract: In Bayesian hierarchical modeling, it is often appealing to allow the conditional density of an (observable or unobservable) random variable $Y$ to change flexibly with categorical and continuous predictors ${\bf X}$. A mixture of regression models is proposed, with the mixture distribution varying with ${\bf X}$. Treating the smoothing parameters and number of mixture components as unknown, the MLE does not exist, motivating an empirical Bayes approach. The proposed method shrinks the spatially-adaptive mixture distributions to a common baseline, while penalizing rapid changes and large numbers of components. The discrete form of the mixture distribution facilitates flexible classification of subjects. A Gibbs sampling algorithm is developed, which embeds a Monte Carlo EM-type stage to estimate smoothing and hyper-parameters. The method is applied to simulated examples and data from an epidemiologic study.

Key words and phrases: Conditional density estimation, Dirichlet process, EM algorithm, Gaussian mixture sieve, Gibbs sampling, nonlinear regression, nonparametric Bayes, smoothing.