Abstract

Bayesian methods are widely employed for variable selection; how

ever, the computational complexity associated with Markov Chain Monte Carlo

(MCMC) techniques often limits their scalability in high-dimensional contexts.

The computation becomes more challenging in mixture models with a substantial

number of latent variables. We propose a variational Bayesian (VB) approach

for high-dimensional structured mixture models to identify important variables

for subgroup analysis. Our method enables efficient and simultaneous variable

selection and parameter estimation by approximating the posterior distribution.

We establish model selection consistency and derive contraction rates for estimation errors, advancing existing VB theoretical results. Additionally, a coordi-

nate ascent variational inference algorithm with data augmentation is developed.

Numerical studies illustrate that our method achieves accuracy comparable to

MCMC while significantly improving computational efficiency. The effectiveness

of our method is validated through real-world applications.

Information

Preprint No.SS-2025-0226
Manuscript IDSS-2025-0226
Complete AuthorsRuqian Zhang, Juan Shen
Corresponding AuthorsJuan Shen
Emailsshenjuan@fudan.edu.cn

References

  1. Anderson, J. L., B. D. Horne, S. M. Stevens, A. S. Grove, S. Barton, Z. P. Nicholas, and et al.
  2. (2007). Randomized trial of genotype-guided versus standard warfarin dosing in patients initiating oral anticoagulation. Circulation 116(22), 2563–2570.
  3. Atchad´e, Y. A. (2017). On the contraction properties of some high-dimensional quasi-posterior distributions. The Annals of Statistics 45(5), 2248 – 2273.
  4. Blei, D. M., A. Kucukelbir, and J. D. McAuliffe (2017). Variational inference: A review for statisticians. Journal of the American Statistical Association 112(518), 859–877.
  5. B¨uhlmann, P. (2013). Statistical significance in high-dimensional linear models. Bernoulli 19(4), 1212–1242.
  6. Cai, T., L. Tian, P. H. Wong, and L. J. Wei (2010). Analysis of randomized comparative clinical trial data for personalized treatment selections. Biostatistics 12(2), 270–282.
  7. Carbonetto, P. and M. Stephens (2012). Scalable variational inference for Bayesian variable selection in regression, and its accuracy in genetic association studies. Bayesian Analysis 7(1), 73–108.
  8. Chen, G., H. Zhong, A. Belousov, and V. Devanarayan (2015). A PRIM approach to predictivesignature development for patient stratification. Statistics in Medicine 34(2), 317–342.
  9. Durante, D. and T. Rigon (2019). Conditionally conjugate mean-field variational Bayes for logistic models. Statistical Science 34(3), 472 – 485.
  10. Gaikwad, T., K. Ghosh, and S. Shetty (2014). Vkorc1 and cyp2c9 genotype distribution in asian countries. Thrombosis Research 134(3), 537–544.
  11. Galy-Fajou, T., F. Wenzel, C. Donner, and M. Opper (2020). Multi-class gaussian process classification made conjugate: Efficient inference via data augmentation. Proceedings of The 35th Uncertainty in Artificial Intelligence Conference 115, 755–765.
  12. Ge, C., B. Lin, and J. S. Liu (2025). A Variational Spike-and-Slab Approach for Group Variable Selection. Bayesian Analysis, 1 – 31.
  13. George, E. I. and R. E. McCulloch (1993). Variable selection via Gibbs sampling. Journal of the American Statistical Association 88(423), 881–889.
  14. Ghosh, J., A. H. Herring, and A. M. Siega-Riz (2011). Bayesian variable selection for latent class models. Biometrics 67(3), 917–925.
  15. Hammer, S. M., K. E. Squires, M. D. Hughes, J. M. Grimes, L. M. Demeter, J. S. Currier, and
  16. et al. (1997). A controlled trial of two nucleoside analogues plus indinavir in persons with human immunodeficiency virus infection and CD4 cell counts of 200 per cubic millimeter or less. New England Journal of Medicine 337(11), 725–733.
  17. Huang, X., Y. Sun, P. Trow, S. Chatterjee, A. Chakravartty, L. Tian, and et al. (2017). Patient subgroup identification for clinical drug development. Statistics in Medicine 36(9), 1414– 1428.
  18. Imai, K. and M. Ratkovic (2013). Estimating treatment effect heterogeneity in randomized program evaluation. The Annals of Applied Statistics 7(1), 443–470.
  19. International Warfarin Pharmacogenetics Consortium (2009). Estimation of the warfarin dose with clinical and pharmacogenetic data. New England Journal of Medicine 360(8), 753– 764.
  20. Italiano, A. (2011). Prognostic or predictive? It’s time to get back to definitions! Journal of Clinical Oncology 29(35), 4718–4718.
  21. Jensen, B. P., P. K. L. Chin, R. L. Roberts, and E. J. Begg (2012). Influence of adult age on the total and free clearance and protein binding of (r)- and (s)-warfarin. British Journal of Clinical Pharmacology 74(5), 797–805.
  22. Khalili, A. and J. Chen (2007). Variable selection in finite mixture of regression models. Journal of the American Statistical Association 102(479), 1025–1038.
  23. Komodromos, M., E. O. Aboagye, M. Evangelou, S. Filippi, and K. Ray (2022). Variational Bayes for high-dimensional proportional hazards models with applications within gene expression. Bioinformatics 38(16), 3918–3926.
  24. Komodromos, M., M. Evangelou, S. Filippi, and K. Ray (2025). Group Spike-and-Slab Variational Bayes. Bayesian Analysis, 1 – 29.
  25. Lee, K. and X. Cao (2021). Bayesian group selection in logistic regression with application to MRI data analysis. Biometrics 77(2), 391–400.
  26. Liu, P., Y. Li, and J. Li (2025). Change surface regression for nonlinear subgroup identification with application to warfarin pharmacogenomics data. Biometrics 81(1), ujae169.
  27. Loh, W.-Y. (2002). Regression trees with unbiased variable selection and interaction detection. Statistica Sinica 12(2), 361–386.
  28. Loh, W.-Y., L. Cao, and P. Zhou (2019). Subgroup identification for precision medicine: A comparative review of 13 methods. WIREs Data Mining and Knowledge Discovery 9(5), e1326.
  29. McLachlan, G. J., S. X. Lee, and S. I. Rathnayake (2019). Finite mixture models. Annual Review of Statistics and Its Application 6, 355–378.
  30. Narisetty, N. N. and X. He (2014). Bayesian variable selection with shrinking and diffusing priors. The Annals of Statistics 42(2), 789 – 817.
  31. Narisetty, N. N., J. Shen, and X. He (2019). Skinny Gibbs: A consistent and scalable Gibbs sampler for model selection. Journal of the American Statistical Association 114(527), 1205–1217.
  32. Pirmohamed, M., G. Burnside, N. Eriksson, A. L. Jorgensen, C. H. Toh, T. Nicholson, and et al.
  33. (2013). A randomized trial of genotype-guided dosing of warfarin. New England Journal of Medicine 369(24), 2294–2303.
  34. Polson, N. G., J. G. Scott, and J. Windle (2013). Bayesian inference for logistic models using P´olya Gamma latent variables. Journal of the American Statistical Association 108(504), 1339–1349.
  35. Ray, K. and B. Szab´o (2022). Variational Bayes for high-dimensional linear regression with sparse priors. Journal of the American Statistical Association 117(539), 1270–1281.
  36. Ray, K., B. Szab´o, and G. Clara (2020). Spike and slab variational Bayes for high dimensional logistic regression. In Advances in Neural Information Processing Systems, Volume 33, pp. 14423–14434.
  37. Sconce, E. A., T. I. Khan, H. A. Wynne, P. Avery, L. Monkhouse, B. P. King, and et al. (2005). The impact of CYP2C9 and VKORC1 genetic polymorphism and patient characteristics upon warfarin dose requirements: proposal for a new dosing regimen. Blood 106(7), 2329– 2333.
  38. Seibold, H., A. Zeileis, and T. Hothorn (2016). Model-based recursive partitioning for subgroup analyses. The International Journal of Biostatistics 12(1), 45–63.
  39. Shen, J. and X. He (2015). Inference for subgroup analysis with a structured logistic-normal mixture model. Journal of the American Statistical Association 110(509), 303–312.
  40. Shen, J. and A. Qu (2020). Subgroup analysis based on structured mixed-effects models for longitudinal data. Journal of Biopharmaceutical Statistics 30(4), 607–622.
  41. Stack, G. and C. B. Maurice (2016). Warfarin pharmacogenetics reevaluated: Subgroup analysis reveals a likely underestimation of the maximum pharmacogenetic benefit by clinical trials. American Journal of Clinical Pathology 145(5), 671–686.
  42. St¨adler, N., P. B¨uhlmann, and S. Van De Geer (2010). ℓ1-penalization for mixture regression models. Test 19(2), 209–256.
  43. Van Dat, N., P. Van Toan, and T. M. Thanh (2022). Solving distribution problems in contentbased recommendation system with gaussian mixture model. Applied Intelligence 52(2), 1602–1614. van der Vliet, R., R. W. Selles, E.-R. Andrinopoulou, R. Nijland, G. M. Ribbers, M. A. Frens,
  44. and et al. (2020). Predicting upper limb motor impairment recovery after stroke: A mixture model. Annals of Neurology 87(3), 383–393.
  45. Wang, Y. (2016). Logistic-normal mixtures with heterogeneous components and high dimensional covariates. Ph. D. thesis, University of Michigan.
  46. Wang, Y. and D. M. Blei (2019). Frequentist consistency of variational Bayes. Journal of the American Statistical Association 114(527), 1147–1161.
  47. Westling, T. and T. H. McCormick (2019). Beyond prediction: A framework for inference with variational approximations in mixture models. Journal of Computational and Graphical Statistics 28(4), 778–789.
  48. Wojnowicz, M. T., S. Aeron, E. L. Miller, and M. Hughes (2022). Easy variational inference for categorical models via an independent binary approximation. Proceedings of the 39th International Conference on Machine Learning 162, 23857–23896.
  49. Yang, Y., D. Pati, and A. Bhattacharya (2020). α-variational inference with statistical guarantees. The Annals of Statistics 48(2), pp. 886–905.
  50. Yang, Y., M. J. Wainwright, and M. I. Jordan (2016). On the computational complexity of high-dimensional Bayesian variable selection. The Annals of Statistics 44(6), 2497 – 2532.
  51. Zhang, C.-X., S. Xu, and J.-S. Zhang (2019). A novel variational Bayesian method for variable selection in logistic regression models. Computational Statistics & Data Analysis 133, 1–19.
  52. Zhang, F. and C. Gao (2020). Convergence rates of variational posterior distributions. The Annals of Statistics 48(4), 2180 – 2207.
  53. Zhang, R., N. N. Narisetty, X. He, and J. Shen (2025). Bayesian variable selection on structured logistic-normal mixture models for subgroup analysis. Electronic Journal of Statistics 19(1), 2876–2922.
  54. Zhao, L., L. Tian, T. Cai, B. Claggett, and L. J. Wei (2013). Effectively selecting a target population for a future comparative study. Journal of the American Statistical Association 108(502), 527–539. Ruqian Zhang

Acknowledgments

This work was partially supported by the National Nature and Science

Foundation of China (11871165, 12331009).

Supplementary Materials

The online supplementary materials contain (1) proofs of the theoretical

results under a known noise variance; (2) extended theoretical results and

proofs under an unknown noise variance; (3) detailed CAVI updates and

their derivation; (4) additional results of simulation studies and sensitivity

analyses; (5) additional information on the real applications.

Supplementary materials are available for download.