Statistica Sinica 19 (2009), 1641-1663

AN INFORMATION MATRIX PRIOR FOR BAYESIAN

ANALYSIS IN GENERALIZED LINEAR MODELS

WITH HIGH DIMENSIONAL DATA

Mayetri Gupta and Joseph G. Ibrahim

Boston University and University of North Carolina

Abstract: An important challenge in analyzing high dimensional data in regression settings is that of facing a situation in which the number of covariates $p$ in the model greatly exceeds the sample size $n$ (sometimes termed the ``$p>n$'' problem). In this article, we develop a novel specification for a general class of prior distributions, called Information Matrix (IM) priors, for high-dimensional generalized linear models. The priors are first developed for settings in which $p<n$, and then extended to the $p>n$ case by defining a ridge parameter in the prior construction, leading to the Information Matrix Ridge (IMR) prior. The IM and IMR priors are based on a broad generalization of Zellner's g-prior for Gaussian linear models. Various theoretical properties of the prior and implied posterior are derived including existence of the prior and posterior moment generating functions, tail behavior, as well as connections to Gaussian priors and Jeffreys' prior. Several simulation studies and an application to a nucleosomal positioning data set demonstrate its advantages over Gaussian, as well as g-priors, in high dimensional settings.

Key words and phrases: Fisher Information, g-prior, importance sampling, model identifiability, prior elicitation.