Statistica Sinica 22 (2012), 1217-1232

HIGH DIMENSIONAL EXPONENTIAL FAMILY

ESTIMATION VIA EMPIRICAL BAYES

Omkar Muralidharan

Stanford University

Abstract: Consider estimating $\theta_{1},\ldots,\theta_{N}$ based on data $z_{i}\sim f_{\theta_{i}}$, where $f_{\theta}$ is a continuous natural exponential family. One successful approach is Empirical Bayes (EB). EB methods assume that $\theta$ come from an unknown prior, and estimate the Bayes procedure corresponding to that prior.

In this paper, we propose a general form of nonparametric EB estimator that uses estimates of the marginal density of $z$ and its derivative. This estimator was first proposed by Zhang (1997) for the normal means problem, $z_{i}\sim\mathcal{N}\left(\theta_{i},1\right)$. We bound the regret of our method in terms of the error in estimating the marginal density and its derivative. As a side point, our proof yields a lower bound on the regret of general estimators.

We illustrate our method in the simultaneous chi-squared problem, where $z_{i}$ is a chi-squared random variable with scale $1/\theta_{i}$. We specialize our theoretical results to this case, and study the empirical performance of our method under different estimators of the marginal and its derivative. Our method outperforms the UMVU estimator and a conjugate prior parametric EB approach.

Key words and phrases: Chi-squared estimation, density estimation, empirical Bayes, exponential family, regret bound, separable estimator.