Generalized AIC for Componentwise Model Selection

Youngjo Lee and Kwok Pui Choi

doi:10.5705/ss.202026.0168

Abstract

A statistical model can be decomposed into a number of interconnected component models.

Model selection can be carried out componentwise to meet each component’s inferential objectives.

We propose a generalized Akaike information criterion that provides a unified framework for optimal

parameter estimation for components of interest while ensuring consistent selection of the remaining

components. This leads to an extension of the definition of degrees of freedom to random parameters.

We illustrate model selection using various information criteria and elucidate correspondences between

Bayesian and likelihood approaches.

Key words and phrases: AIC; BIC; CAIC; DIC; Lasso; Degrees of freedom 1

Information

Preprint No.	SS-2026-0168
Manuscript ID	SS-2026-0168
Complete Authors	Youngjo Lee, Kwok Pui Choi
Corresponding Authors	Kwok Pui Choi
Emails	stackp@nus.edu.sg

References

Akaike, H. (1973). Information theory as an extension of the maximum likelihood principle. In Second International Symposium on Information Theory, pp. 276–281. (B.N. Petrov and F. Csaki, ed.) Akademiai Kiado, Budapest, Birnbaum,
A. (1962). On the foundations of statistical inference. Journal of the American Statistical Association 57(298), 269–306.
Bjørnstad, J. F. (1996). On the generalization of the likelihood function and the likelihood principle. Journal of the American Statistical Association 91(434), 791–806.
Breslow, N. and D. Clayton (1993). Generalized linear mixed models. Journal of the American Statistical Association 88, 9–25.
Cox, D. R. and N. Reid (1987). Parameter orthogonality and approximate conditional inference. Journal of the Royal Statistical Society Series B: Statistical Methodology 49(1), 1–39.
Donohue, M., R. Overholser, R. Xu, and F. Vaida (2011). Conditional akaike information under generalized linear and proportional hazards mixed models. Biometrika 98(3), 685–700.
Glidden, D. V. and K. Y. Liang (2002). Ascertainment adjustment in complex diseases. Genetic Epidemiology: The Official Publication of the International Genetic Epidemiology Society 23(3), 201–208.
Ha, I. D. and Y. Lee (2003). Estimating frailty models via poisson hierarchical generalized linear models. Journal of Computational and Graphical Statistics 12(3), 663–681.
Ha, I. D., Y. Lee, and G. MacKenzie (2007). Model selection for multi-component frailty models. Statistics in Medicine 26(26), 4790–4807.
Han, J. and Y. Lee (2024). Enhanced Laplace approximation. Journal of Multivariate Analysis 202, 105321.
Hastie, T. J. and R. J. Tibshirani (1990). Generalized Additive Models, Volume 43 of Monographs on Statistics and Applied Probability. Chapman & Hall.
Henderson, C. R., O. Kempthorne, S. R. Searle, and C. Von Krosigk (1959). The estimation of environmental and genetic trends from records subject to culling. Biometrics 15(2), 192–218.
Kaymaz, O., D. Vionanda, F. Dogru, Y. Lee, H. Wood and A. Gusnanto (2026). Sparse logistic regression on genomic d f di i f h l i l b T i IEEE/ACM T i C i l Biology and Bioinformatics.
Lee, D., W. Lee, Y. Lee and Y. Pawitan (2010). Super sparse principal analysis for high-throughput genomic data. BMC Bioinformatics, 11(1), 296.
Lee, D., W. Lee, Y. Lee and Y. Pawitan (2011). Sparse partial least-squares regression and its applications to high-throughput data analysis. Chemometrics and Intelligent Laboratory Systems, 109(1), 1–8.
Lee, Y. (2001). Can we recover information from concordant pairs in binary matched pairs? Journal of Applied Statistics 28(2), 239–246.
Lee, Y. (2026). Resolving the induction problem: Can we state with complete confidence that the sun rises forever? To appear in International Statistical Review.
Lee, H. and Y. Lee (2025). Statistical inference for random unknowns via modifications of extended likelihood. arXiv: 2310.09955v3.
Lee, Y. and J. A. Nelder (1996). Hierarchical generalized linear models. Journal of the Royal Statistical Society Series B: Statistical Methodology 58(4), 619–656.
Lee, Y. and J. A. Nelder (2001). Hierarchical generalised linear models: a synthesis of generalised linear models, random-effect models and structured dispersions. Biometrika 88(4), 987–1006.
Lee, Y. and J. A. Nelder (2004). Conditional and marginal models: another view. Statistical Science 19(2), 219–238.
Lee, Y. and J. A. Nelder (2006a). Double hierarchical generalized linear models (with discussion). Journal of the
Royal Statistical Society Series C: Applied Statistics 55(2), 139–185.
Lee, Y. and J. A. Nelder (2006b). Fitting via alternative random-effect models. Statistics and Computing 16, 69–75.
Lee, Y., J. A. Nelder, and Y. Pawitan (2017). Generalized linear models with random effects: unified analysis via H-likelihood. Chapman and Hall/CRC.
Lee, Y. and M. Noh (2026). On the use of Gaussian process and deep neural network for semi-parametric Poisson
Lee, Y. and H.-S. Oh (2014). A new sparse variable selection via random-effect model. Journal of Multivariate Analysis 125, 89–99.
Liang, H., H. Wu, and G. Zou (2008). A note on conditional AIC for linear mixed-effects models. Biometrika 95(3), 773–778.
Meng, X.-L. (2009). Decoding the h-likelihood. Statistical Science 24(3), 280–293.
Neyman, J. and E. L. Scott (1948). Consistent estimates based on partially consistent observations. Econometrica: Journal of the Econometric Society, 1–32.
Ng, C. T., W. Lee and Y. Lee (2018). Change-point estimators with true identification property. Bernoulli, 24(1), 616–660.
Noh, M. and Y. Lee (2007). Robust modeling for inference from generalized linear model classes. Journal of the American Statistical Association 102(479), 1059–1072.
Noh, M., Y. Lee, and Y. Pawitan (2005). Robust ascertainment-adjusted parameter estimation. Genetic Epidemiology: The Official Publication of the International Genetic Epidemiology Society 29(1), 68–75.
Park, T. and G. Casella (2008). The Bayesian Lasso. Journal of the American Statistical Association 103(482), 681– 686.
Patterson, H. D. and R. Thompson (1971). Recovery of inter-block information when block sizes are unequal. Biometrika 58(3), 545–554.
Schwarz, G. (1978). Estimating the dimension of a model. The Annals of Statistics 6(2), 461–464.
Smyth, G. K. and A. P. Verbyla (1996). A conditional likelihood approach to residual maximum likelihood estimation in generalized linear models. Journal of the Royal Statistical Society Series B: Statistical Methodology 58(3), 565–572.
Spiegelhalter, D. J., N. G. Best, B. P. Carlin, and A. Van Der Linde (2002). Bayesian measures of model complexity d fi J l f h R l S i i l S i S i B S i i l M h d l 64(4) 583 639
Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society Series B: Statistical Methodology 58(1), 267–288.
Vaida, F. and S. Blanchard (2005). Conditional Akaike information for mixed effects models. Biometrika 92, 351–370.
Yang, Y. (2005). Can the strengths of AIC and BIC be shared? A conflict between model indentification and regression estimation. Biometrika 92(4), 937–950.
Yates, F. (1940). The recovery of inter-block information in balanced incomplete block designs. Annals of Eugenics 10(1), 317–325.
Yu, D. and K. K. Yau (2012). Conditional akaike information criterion for generalized linear mixed models. Computational Statistics & Data Analysis 56(3), 629–644.
Zeng, Y., Z. Wang, Y. Lee, and N. Tang (2025). Simultaneous outlier detection and prediction for kriging with true identification. Journal of Computational and Graphical Statistics, 1–12.

Acknowledgments

We thank the Associate Editor and the two anonymous reviewers for their careful reading of

the manuscript and for their constructive and helpful comments, which have greatly improved

the paper.

Supplementary Materials

This online supplementary material consists of the following: Proofs of Theorems 1–3, Proposition 2, detailed description of the four real data examples and Table A for the components

ψ and ς.

Supplementary materials are available for download.

[1] Akaike, H. (1973). Information theory as an extension of the maximum likelihood principle. In Second International Symposium on Information Theory, pp. 276–281. (B.N. Petrov and F. Csaki, ed.) Akademiai Kiado, Budapest, Birnbaum,

[2] A. (1962). On the foundations of statistical inference. Journal of the American Statistical Association 57(298), 269–306.

[3] Bjørnstad, J. F. (1996). On the generalization of the likelihood function and the likelihood principle. Journal of the American Statistical Association 91(434), 791–806.

[4] Breslow, N. and D. Clayton (1993). Generalized linear mixed models. Journal of the American Statistical Association 88, 9–25.

[5] Cox, D. R. and N. Reid (1987). Parameter orthogonality and approximate conditional inference. Journal of the Royal Statistical Society Series B: Statistical Methodology 49(1), 1–39.

[6] Donohue, M., R. Overholser, R. Xu, and F. Vaida (2011). Conditional akaike information under generalized linear and proportional hazards mixed models. Biometrika 98(3), 685–700.

[7] Glidden, D. V. and K. Y. Liang (2002). Ascertainment adjustment in complex diseases. Genetic Epidemiology: The Official Publication of the International Genetic Epidemiology Society 23(3), 201–208.

[8] Ha, I. D. and Y. Lee (2003). Estimating frailty models via poisson hierarchical generalized linear models. Journal of Computational and Graphical Statistics 12(3), 663–681.

[9] Ha, I. D., Y. Lee, and G. MacKenzie (2007). Model selection for multi-component frailty models. Statistics in Medicine 26(26), 4790–4807.

[10] Han, J. and Y. Lee (2024). Enhanced Laplace approximation. Journal of Multivariate Analysis 202, 105321.

[11] Hastie, T. J. and R. J. Tibshirani (1990). Generalized Additive Models, Volume 43 of Monographs on Statistics and Applied Probability. Chapman & Hall.

[12] Henderson, C. R., O. Kempthorne, S. R. Searle, and C. Von Krosigk (1959). The estimation of environmental and genetic trends from records subject to culling. Biometrics 15(2), 192–218.

[13] Kaymaz, O., D. Vionanda, F. Dogru, Y. Lee, H. Wood and A. Gusnanto (2026). Sparse logistic regression on genomic d f di i f h l i l b T i IEEE/ACM T i C i l Biology and Bioinformatics.

[14] Lee, D., W. Lee, Y. Lee and Y. Pawitan (2010). Super sparse principal analysis for high-throughput genomic data. BMC Bioinformatics, 11(1), 296.

[15] Lee, D., W. Lee, Y. Lee and Y. Pawitan (2011). Sparse partial least-squares regression and its applications to high-throughput data analysis. Chemometrics and Intelligent Laboratory Systems, 109(1), 1–8.

[16] Lee, Y. (2001). Can we recover information from concordant pairs in binary matched pairs? Journal of Applied Statistics 28(2), 239–246.

[17] Lee, Y. (2026). Resolving the induction problem: Can we state with complete confidence that the sun rises forever? To appear in International Statistical Review.

[18] Lee, H. and Y. Lee (2025). Statistical inference for random unknowns via modifications of extended likelihood. arXiv: 2310.09955v3.

[19] Lee, Y. and J. A. Nelder (1996). Hierarchical generalized linear models. Journal of the Royal Statistical Society Series B: Statistical Methodology 58(4), 619–656.

[20] Lee, Y. and J. A. Nelder (2001). Hierarchical generalised linear models: a synthesis of generalised linear models, random-effect models and structured dispersions. Biometrika 88(4), 987–1006.

[21] Lee, Y. and J. A. Nelder (2004). Conditional and marginal models: another view. Statistical Science 19(2), 219–238.

[22] Lee, Y. and J. A. Nelder (2006a). Double hierarchical generalized linear models (with discussion). Journal of the

[23] Royal Statistical Society Series C: Applied Statistics 55(2), 139–185.

[24] Lee, Y. and J. A. Nelder (2006b). Fitting via alternative random-effect models. Statistics and Computing 16, 69–75.

[25] Lee, Y., J. A. Nelder, and Y. Pawitan (2017). Generalized linear models with random effects: unified analysis via H-likelihood. Chapman and Hall/CRC.

[26] Lee, Y. and M. Noh (2026). On the use of Gaussian process and deep neural network for semi-parametric Poisson

[27] Lee, Y. and H.-S. Oh (2014). A new sparse variable selection via random-effect model. Journal of Multivariate Analysis 125, 89–99.

[28] Liang, H., H. Wu, and G. Zou (2008). A note on conditional AIC for linear mixed-effects models. Biometrika 95(3), 773–778.

[29] Meng, X.-L. (2009). Decoding the h-likelihood. Statistical Science 24(3), 280–293.

[30] Neyman, J. and E. L. Scott (1948). Consistent estimates based on partially consistent observations. Econometrica: Journal of the Econometric Society, 1–32.

[31] Ng, C. T., W. Lee and Y. Lee (2018). Change-point estimators with true identification property. Bernoulli, 24(1), 616–660.

[32] Noh, M. and Y. Lee (2007). Robust modeling for inference from generalized linear model classes. Journal of the American Statistical Association 102(479), 1059–1072.

[33] Noh, M., Y. Lee, and Y. Pawitan (2005). Robust ascertainment-adjusted parameter estimation. Genetic Epidemiology: The Official Publication of the International Genetic Epidemiology Society 29(1), 68–75.

[34] Park, T. and G. Casella (2008). The Bayesian Lasso. Journal of the American Statistical Association 103(482), 681– 686.

[35] Patterson, H. D. and R. Thompson (1971). Recovery of inter-block information when block sizes are unequal. Biometrika 58(3), 545–554.

[36] Schwarz, G. (1978). Estimating the dimension of a model. The Annals of Statistics 6(2), 461–464.

[37] Smyth, G. K. and A. P. Verbyla (1996). A conditional likelihood approach to residual maximum likelihood estimation in generalized linear models. Journal of the Royal Statistical Society Series B: Statistical Methodology 58(3), 565–572.

[38] Spiegelhalter, D. J., N. G. Best, B. P. Carlin, and A. Van Der Linde (2002). Bayesian measures of model complexity d fi J l f h R l S i i l S i S i B S i i l M h d l 64(4) 583 639

[39] Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society Series B: Statistical Methodology 58(1), 267–288.

[40] Vaida, F. and S. Blanchard (2005). Conditional Akaike information for mixed effects models. Biometrika 92, 351–370.

[41] Yang, Y. (2005). Can the strengths of AIC and BIC be shared? A conflict between model indentification and regression estimation. Biometrika 92(4), 937–950.

[42] Yates, F. (1940). The recovery of inter-block information in balanced incomplete block designs. Annals of Eugenics 10(1), 317–325.

[43] Yu, D. and K. K. Yau (2012). Conditional akaike information criterion for generalized linear mixed models. Computational Statistics & Data Analysis 56(3), 629–644.

[44] Zeng, Y., Z. Wang, Y. Lee, and N. Tang (2025). Simultaneous outlier detection and prediction for kriging with true identification. Journal of Computational and Graphical Statistics, 1–12.