Abstract

Composite likelihood usually ignores dependencies among response

components, while variational approximation to likelihood ignores dependencies

among parameter components. What both methods have in common is that they

essentially break the dependence of random effects. In this paper, we derive a

Gaussian variational approximation to the composite log-likelihood function for

Poisson and Gamma models with crossed random effects. We present theoretical

aspects of the estimates derived from this approximation and support these theories with simulation studies. Specifically, we show the estimates are consistent

with a convergence rate m−1/2+n−1/2, where m and n denote the number of rows

and columns, respectively. We further provide detailed asymptotic normality results under a new regime where log m/ log n →δ for δ ∈(1/2, 2). Additional sim-

ulation studies show that our method yields comparable estimation performance

and is slightly faster than the Laplace approximation in the package glmmTMB and

a Gaussian variational approximation to the full log-likelihood function.

Information

Preprint No.SS-2024-0346
Manuscript IDSS-2024-0346
Complete AuthorsLibai Xu, Nancy Reid, Dehan Kong
Corresponding AuthorsLibai Xu
Emailslbxu@suda.edu.cn

References

  1. Adam, F., A. Kurnia, I. Purnaba, I. Mangku, and A. Soleh (2021). Modeling the amount of insurance claim using Gamma linear mixed model with AR (1) random effect. In Journal of Physics: Conference Series, Volume 1863, pp. 012027. IOP Publishing.
  2. Baayen, R. H., D. J. Davidson, and D. M. Bates (2008). Mixed-effects modeling with crossed random effects for subjects and items. Journal of Memory and Language 59(4), 390–412.
  3. Bartolucci, F., F. Chiaromonte, P. K. Don, and B. G. Lindsay (2017). Composite likelihood inference in a discrete latent variable model for two-way “clustering-by-segmentation” problems. Journal of Computational and Graphical Statistics 26(2), 388–402.
  4. Bartolucci, F. and M. Lupparelli (2016). Pairwise likelihood inference for nested hidden Markov chain models for multilevel longitudinal data. Journal of the American Statistical Association 111(513), 216–228.
  5. Bellio, R., S. Ghosh, A. B. Owen, and C. Varin (2025). Consistent and scalable composite GVACL for crossed random effects models likelihood estimation of probit models with crossed random effects. Biometrika 112(3), asaf037.
  6. Bellio, R. and C. Varin (2005). A pairwise likelihood approach to generalized linear models with crossed random effects. Statistical Modelling 5(3), 217–227.
  7. Blei, D. M., A. Kucukelbir, and J. D. McAuliffe (2017). Variational inference: A review for statisticians. Journal of the American Statistical Association 112(518), 859–877.
  8. Breslow, N. E. and D. G. Clayton (1993). Approximate inference in generalized linear mixed models. Journal of the American Statistical Association 88(421), 9–25.
  9. Brooks, M. E., K. Kristensen, K. J. Van Benthem, A. Magnusson, C. W. Berg, A. Nielsen, H. J.
  10. Skaug, M. Machler, and B. M. Bolker (2017). glmmTMB balances speed and flexibility among packages for zero-inflated generalized linear mixed modeling. The R Journal 9(2), 378–400.
  11. Chung, S. and L. Cai (2021). Cross-classified random effects modeling for moderated item calibration. Journal of Educational and Behavioral Statistics 46(6), 651–681.
  12. Coull, B. A., J. P. Hobert, L. M. Ryan, and L. B. Holmes (2001). Crossed random effect models for multiple outcomes in a study of teratogenesis. Journal of the American Statistical Association 96(456), 1194–1204.
  13. Ghandwani, D., S. Ghosh, T. Hastie, and A. Owen (2025). Scalable solutions for crossed randomeffect models with random slopes. Electronic Journal of Statistics 19(2), 5368–5408.
  14. Ghosh, S., T. Hastie, and A. B. Owen (2022a). Backfitting for large scale crossed random effects regressions. The Annals of Statistics 50(1), 560–583.
  15. Ghosh, S., T. Hastie, and A. B. Owen (2022b). Scalable logistic regression with crossed random effects. Electronic Journal of Statistics 16(2), 4604–4635.
  16. Goplerud, M., O. Papaspiliopoulos, and G. Zanella (2025). Partially factorized variational inference for high-dimensional mixed models. Biometrika 112(2), asae067.
  17. Hall, P., J. T. Ormerod, and M. P. Wand (2011). Theory of Gaussian variational approximation for a Poisson mixed model. Statistica Sinica 21, 369–389.
  18. Hall, P., T. Pham, M. P. Wand, and S. S. Wang (2011). Asymptotic normality and valid inference for Gaussian variational approximation. The Annals of Statistics 39(5), 2502–2532. GVACL for crossed random effects models
  19. Hui, F. K., D. I. Warton, J. T. Ormerod, V. Haapaniemi, and S. Taskinen (2017). Variational approximations for generalized linear latent variable models. Journal of Computational and Graphical Statistics 26(1), 35–43.
  20. Hui, F. K., C. You, H. L. Shang, and S. M¨uller (2019). Semiparametric regression using variational approximations. Journal of the American Statistical Association 114, 1765– 1777.
  21. Jeon, M., F. Rijmen, and S. Rabe-Hesketh (2017). A variational maximization–maximization algorithm for generalized linear mixed models with crossed random effects. Psychometrika 82(3), 693–716.
  22. Lin, X. and N. E. Breslow (1996). Bias correction in generalized linear mixed models with multiple components of dispersion. Journal of the American Statistical Association 91(435), 1007–1016.
  23. Liu, Q. and D. A. Pierce (1994). A note on Gauss–Hermite quadrature. Biometrika 81(3), 624–629.
  24. McCulloch, C. E. (1997). Maximum likelihood algorithms for generalized linear mixed models. Journal of the American Statistical Association 92(437), 162–170.
  25. Menictas, M., G. Di Credico, and M. P. Wand (2023). Streamlined variational inference for linear mixed models with crossed random effects. Journal of Computational and Graphical Statistics 32(1), 99–115.
  26. Ormerod, J. T. and M. P. Wand (2012). Gaussian variational approximate inference for generalized linear mixed models. Journal of Computational and Graphical Statistics 21(1), 2–17.
  27. Renard, D., G. Molenberghs, and H. Geys (2004). A pairwise likelihood approach to estimation in multilevel probit models. Computational Statistics & Data Analysis 44(4), 649–667.
  28. Rijmen, F. and M. Jeon (2013). Fitting an item response theory model with random item effects across groups by a variational approximation method. Annals of Operations Research 206(1), 647–662. Schall, R.
  29. (1991). Estimation in generalized linear models with random effects. Biometrika 78(4), 719–727.
  30. Shi, X., X.-S. Wang, and A. Wong (2022). Explicit Gaussian variational approximation for the GVACL for crossed random effects models Poisson lognormal mixed model. Mathematics 10(23), 4542:1–18.

Acknowledgments

The authors are grateful to Dr. Fia Fridayanti Adam for sharing the motor

vehicle insurance data, and to the anonymous referees, associate editor, and

editor for their careful reading and valuable comments, which greatly improved the overall quality of our original manuscript. This research was par-

tially supported by the Natural Sciences and Engineering Research Council

of Canada, the Natural Science Foundation of Jiangsu Province (Grant No.

BK20250825), and the Natural Science Foundation of the Jiangsu Higher

Education Institutions of China (Grant No. 24KJB110024).

Supplementary Materials

The online Supplementary Material includes proofs of Theorems 1 and

2, additional simulation results, and additional analyses of the insurance

data. The R code for simulations is available at https://github.com/libaixu-

GVACL for crossed random effects modelsREFERENCES

Supplementary materials are available for download.