Abstract

We propose a unified framework to draw inferences for regression coefficients

in a generalized linear model (GLM) following Lasso-based variable selection. We adapt

to non-Gaussian GLMs a recently developed parametric programming strategy for postselection inference in the linear model with a Gaussian response by drawing parallels

between maximum likelihood estimation in GLMs and least squares estimation in linear

models. We then conduct post-selection inference based on a linearized model for pseudo

response and covariate data strategically created based on the raw data. Using synthetic

data generated from regression models for three different types of non-Gaussian responses

in simulation experiments, we demonstrate that the proposed method effectively corrects

the naive inference that ignores variable selection while achieving greater efficiency than a

polyhedral-based post-selection adjustment.

Key words and phrases: beta regression, Lasso, logistic regression, Poisson regression, selection event 1 1

Information

Preprint No.SS-2025-0194
Manuscript IDSS-2025-0194
Complete AuthorsQinyan Shen, Karl Gregory, Xianzheng Huang
Corresponding AuthorsKarl Gregory
Emailsgregorkb@stat.sc.edu

References

  1. Albert, A. and J. A. Anderson (1984). On the existence of maximum likelihood estimates in logistic regression models. Biometrika 71(1), 1–10.
  2. Bachoc, F., H. Leeb, and B. M. Pötscher (2019). Valid confidence intervals for post-model-selection pre36 dictors. The Annals of Statistics 47(3), 1475–1504.
  3. Bachoc, F., D. Preinerstorfer, and L. Steinberger (2020). Uniformly valid confidence intervals post-modelselection. The Annals of Statistics 48(1), 440–463.
  4. Berk, R., L. Brown, A. Buja, K. Zhang, L. Zhao, et al. (2013). Valid post-selection inference. The Annals of Statistics 41(2), 802–837.
  5. Bhatia, R., S. Yadav, R. Sharma, Shubneet, A. R. Yadav, and N. S. Talwandi (2025). A comparative study of feature selection techniques for predicting student academic performance using educational data. In International Conference on Data Analytics & Management, pp. 352–362. Springer.
  6. Davison, A. C. (2003). Statistical models, Volume 11. Cambridge university press.
  7. Ferrari, S. and F. Cribari-Neto (2004). Beta regression for modelling rates and proportions. Journal of applied statistics 31(7), 799–815.
  8. Gurmu, S. (1997). Semi-parametric estimation of hurdle regression models with an application to Medicaid utilization. Journal of Applied Econometrics 12(3), 225–242.
  9. Hopkins, M., E. Reeber, G. Forman, and J. Suermondt (1999). Spambase. UCI Machine Learning Repository. DOI: https://doi.org/10.24432/C53G6X.
  10. Huber, P. J. (2011). Robust statistics. In International encyclopedia of statistical science, pp. 1248–1251. Springer.
  11. Hussain, S. (2018). Student Academics Performance. UCI Machine Learning Repository. DOI: https://doi.org/10.24432/C50W30.
  12. Kesgin, K., S. Kiraz, S. Kosunalp, and B. Stoycheva (2025). Beyond performance: Explaining and ensuring fairness in student academic performance prediction with machine learning. Applied Sciences 15(15), 8409.
  13. Kivaranovic, D. and H. Leeb (2021). On the length of post-model-selection confidence intervals conditional on polyhedral constraints. Journal of the American Statistical Association 116(534), 845–857.
  14. Kleiber, C. and A. Zeileis (2008). Applied Econometrics with R. New York: Springer-Verlag.
  15. Kuchibhotla, A. K., L. D. Brown, A. Buja, J. Cai, E. I. George, and L. H. Zhao (2020). Valid post-selection inference in model-free linear regression. The Annals of Statistics 48(5), 2953 – 2981.
  16. Le Duy, V. N. and I. Takeuchi (2021). Parametric programming approach for more powerful and general lasso selective inference. International conference on artificial intelligence and statistics, 901–909.
  17. Lee, J. D., D. L. Sun, Y. Sun, J. E. Taylor, et al. (2016). Exact post-selection inference, with application to the lasso. Annals of Statistics 44(3), 907–927.
  18. Lee, J. D., Y. Sun, and J. E. Taylor (2015). On model selection consistency of regularized M-estimators. Electronic Journal of Statistics 9(1), 608 – 642.
  19. McCullagh, P. and J. A. Nelder (1989). Generalized Linear Models, Volume 37. CRC Press.
  20. Meinshausen, N., L. Meier, and P. Bühlmann (2009). P-values for high-dimensional regression. Journal of the American Statistical Association 104(488), 1671–1681.
  21. Neufeld, A. C., L. L. Gao, and D. M. Witten (2022). Tree-values: selective inference for regression trees. The Journal of Machine Learning Research 23(1), 13759–13801.
  22. Panigrahi, S. and J. Taylor (2023). Approximate selective inference via maximum likelihood. Journal of the American Statistical Association 118(544), 2810–2820.
  23. Pirenne, S. and G. Claeskens (2024). Parametric programming-based approximate selective inference for adaptive lasso, adaptive elastic net and group lasso. Journal of Statistical Computation and Simulation 94(11), 2412–2435.
  24. Prerika, Nitika, and K. Kumar (2025). Email spam detection using artificial neural network with hybrid feature selection. In L. Garg, N. Kesswani, and I. Brigui (Eds.), AI Technologies for Information Systems and Management Science, Cham, pp. 108–117. Springer Nature Switzerland.
  25. Rasines, D. G. and G. A. Young (2023). Splitting strategies for post-selection inference. Biometrika 110(3), 597–614.
  26. Rinaldo, A., L. Wasserman, M. G’Sell, et al. (2019). Bootstrapping and sample splitting for highdimensional, assumption-lean inference. Annals of Statistics 47(6), 3438–3469.
  27. Shen, Q., K. Gregory, and X. Huang (2024). Post-selection inference in regression models for group testing data. Biometrics 80(3), ujae101.
  28. Taylor, J. and R. Tibshirani (2018). Post-selection inference for-penalized likelihood models. Canadian Journal of Statistics 46(1), 41–61.
  29. Tibshirani, R. J., J. Taylor, R. Lockhart, and R. Tibshirani (2016). Exact post-selection inference for sequential regression procedures. Journal of the American Statistical Association 111(514), 600– 620.
  30. Wainwright, M. J. (2009). Sharp thresholds for high-dimensional and noisy sparsity recovery using l1constrained quadratic programming (lasso). IEEE transactions on information theory 55(5), 2183– 2202.
  31. Wasserman, L. and K. Roeder (2009). High dimensional variable selection. Annals of statistics 37(5A), 2178–2201.
  32. White, H. (1982). Maximum likelihood estimation of misspecified models. Econometrica: Journal of the Econometric Society 50, 1–25.
  33. Zhang, X. and G. Cheng (2017). Simultaneous inference for high-dimensional linear models. Journal of the American Statistical Association 112(518), 757–768.
  34. Zhao, P. and B. Yu (2006). On model selection consistency of lasso. The Journal of Machine Learning Research 7, 2541–2563.
  35. Zhao, Q., D. S. Small, and A. Ertefaie (2022). Selective inference for effect modification via the lasso. Journal of the Royal Statistical Society Series B: Statistical Methodology 84(2), 382–413. ---