Statistica Sinica 22 (2012), 755-776

doi:http://dx.doi.org/10.5705/ss.2010.090

A FINITE MIXTURE MODEL FOR WORKING

CORRELATION MATRICES

IN GENERALIZED ESTIMATING EQUATIONS

Lili Xu$^1$, Nan Lin$^2$, Baoxue Zhang$^{1}$ and Ning-Zhong Shi$^1$

$^1$Northeast Normal University and $^2$Washington University in St. Louis

Abstract: The generalized estimating equations (GEE) method has been widely used to analyze longitudinal data since it was proposed by Liang and Zeger (1986). It is well known that the efficiency of the GEE estimator can be seriously affected by the choice of the working correlation matrix. To address the associated misspecification issue, we propose an estimator called mix-GEE based on a finite mixture model for the working correlation. Under mild regularity conditions, the mix-GEE estimator is consistent, asymptotically normal, and asymptotically efficient if data are from a Gaussian mixture model. An important feature of the mix-GEE method is that it guarantees the positive definiteness of the estimated working correlation matrix if either the AR(1) or exchangeable structure is included. It is numerically more stable and displays a better finite sample efficiency than the hybrid GEE method (Leung, Wang and Zhu (2009)). The value of our method is further demonstrated by simulation studies and data examples.

Key words and phrases: GEE, longitudinal data, working correlation, misspecification, finite mixture, positive definite, pseudo-likelihood, PL-EM algorithm.