Statistica Sinica 11(2001), 1005-1029

AUTOMATIC BAYESIAN MODEL AVERAGING FOR

LINEAR REGRESSION AND APPLICATIONS IN

BAYESIAN CURVE FITTING

Faming Liang$^\dagger$, Young K Truong$^\dagger$ and Wing Hung Wong$^\ddagger$

$^\dagger$The National University of Singapore and $^\ddagger$Harvard School of Public Health

Abstract: With the development of MCMC methods, Bayesian methods play a more and more important role in model selection and statistical prediction. However, the sensitivity of the methods to prior distributions has caused much difficulty to users. In the context of multiple linear regression, we propose an automatic prior setting, in which there is no parameter to be specified by users. Under the prior setting, we show that sampling from the posterior distribution is approximately equivalent to sampling from a Boltzmann distribution defined on $C_p$ values. The numerical results show that the Bayesian model averaging procedure resulted from the automatic prior settin provides a significant improvement in predictive performance over other two procedures proposed in the literature. The procedure is extended to the problem of Bayesian curve fitting with regression splines. Evolutionary Monte Carlo is used to sample from the posterior distributions.

Key words and phrases: Bayesian model averaging, curve fitting, evolutionary Monte Carlo, Mallows' Cp, Markov chain Monte Carlo.