Abstract: This paper proposes a Bayesian method to estimate shape-restricted functions using Gaussian process priors. The proposed model enforces shape-restrictions by assuming that the derivatives of the functions are squares of Gaussian processes. The resulting functions, after integration, are monotonic, monotonic convex or concave, U–Shaped, and S–shaped. The latter two allow estimation of extreme points and inflection points. The Gaussian process’s covariance function has hyper parameters to control the smoothness of the function and the tradeoff between the data and the prior distribution. The Bayesian analysis of these hyper parameters provides a data–driven method to identify the appropriate amount of smoothing. The posterior distributions of the proposed models are consistent. We modify the basic model with a spike-and-slab prior that improves model fit when the true function is on the boundary of the constraint space. We also examine Bayesian hypothesis testing for shape restrictions and discuss its potentials and limitations. We contrast our approach with existing Bayesian regression models with monotonicity and concavity and illustrate the empirical performance of the proposed models with synthetic and actual data.

Key words and phrases: Adaptive Markov chain Monte Carlo, isotonic regression, Karhunen-Lo`eve expansion, lasso, model choice, semiparametric regression, shape restriction, smoothing, spectral representation.