Abstract

We present a novel Bayesian approach for high-dimensional grouped regression

under sparsity. We leverage a sparse projection method that uses a sparsity-inducing map

to induce a posterior on a lower-dimensional parameter space. Our method introduces

three distinct projection maps based on popular penalty functions: the Group LASSO

projection-posterior, the Group SCAD projection-posterior, and the Adaptive Group

LASSO projection-posterior. Each projection map is constructed to immerse posterior

samples into a structured, sparse space, allowing for effective group selection and estimation in high-dimensional settings. We derive optimal posterior contraction rates for esti-

mation and prediction, thereby proving that the methods are model-selection consistent.

We also propose a Debiased Group LASSO Projection Map that ensures correct asymptotic coverage of credible sets. Our methodology is particularly suited for applications

in nonparametric additive models, where we use B-spline expansions to capture complex relationships between covariates and the response. Extensive simulations validate

our theoretical findings, demonstrating the robustness of our approach across different

settings. Finally, we illustrate the practical utility of our method with an application to

brain MRI volume data from the Alzheimer’s Disease Neuroimaging Initiative (ADNI),

where our model identifies key brain regions associated with Alzheimer’s Disease severity.

Key words and phrases: Grouped-regression; Sparse; High-dimension; Coverage; Penalty; Projection

Information

Preprint No.SS-2025-0071
Manuscript IDSS-2025-0071
Complete AuthorsSamhita Pal, Subhashis Ghosal
Corresponding AuthorsSamhita Pal
Emailssamhitapal3896@gmail.com

References

  1. S Derin Babacan, Shinichi Nakajima, and Minh N Do. Bayesian group-sparse modeling and variational inference. IEEE Transactions on Signal Processing, 62(11):2906–2921, 2014.
  2. Prithwish Bhaumik and Subhashis Ghosal. Bayesian two-step estimation in differential equation models. Electronic Journal of Statistics, 9(2):3124–3154, 2015.
  3. Peter J Bickel, Ya’acov Ritov, and Alexandre B Tsybakov. Simultaneous analysis of Lasso and Dantzig selector. The Annals of Statistics, 37(4):1705–1732, 2009.
  4. Peter B¨uhlmann and Sara van de Geer. Statistics for High-Dimensional Data: Methods, Theory and Applications. Springer Science & Business Media, 2011.
  5. George Casella, Malay Ghosh, Jeff Gill, and Minjung Kyung. Penalized regression, standard errors, and Bayesian lassos. Bayesian Analysis, 5(2):369–411, 2010.
  6. Moumita Chakraborty and Subhashis Ghosal. Coverage of credible intervals in nonparametric monotone regression. The Annals of Statistics, 49(2):1011–1028, 2021.
  7. Jianqing Fan and Runze Li. Variable selection via nonconcave penalized likelihood and its oracle properties.
  8. Journal of the American Statistical Association, 96(456):1348–1360, 2001.
  9. Alessio Farcomeni. Bayesian constrained variable selection. Statistica Sinica, 20:1043–1062, 2010.
  10. Edward I George and Robert E McCulloch. Stochastic search variable selection. Markov chain Monte Carlo in Practice, 68(1):203–214, 1995.
  11. Xiao Guo, Hai Zhang, Yao Wang, and Jiang-Lun Wu. Model selection and estimation in high dimensional regression models with group scad. Statistics & Probability Letters, 103:
  12. 86–92, 2015.
  13. Toshio Honda. The de-biased group lasso estimation for varying coefficient models. Annals of the Institute of Statistical Mathematics, 73:3–29, 2021.
  14. Jian Huang and Huiliang Xie. Asymptotic oracle properties of SCAD-penalized least squares estimators. In Asymptotics: Particles, Processes and Inverse Problems (Cator, E. et al., eds.), volume 55 of IMS Lecture Notes Monograph Series, pages 149–166. Institute of
  15. Mathematical Statistics, Beachwood, OH, 2007.
  16. Jian Huang, Shuange Ma, Huiliang Xie, and Cun-Hui Zhang. A group bridge approach for variable selection. Biometrika, 96(2):339–355, 2009.
  17. Jian Huang, Joel L Horowitz, and Fengrong Wei. Variable selection in nonparametric additive models. The Annals of Statistics, 38(4):2282–2313, 2010.
  18. Jian Huang, Patrick Breheny, and Shuangge Ma. A selective review of group selection in highdimensional models. Statistical Science: a Review Journal of the Institute of Mathematical Statistics, 27(4), 2012. Kei M Igarashi.
  19. Entorhinal cortex dysfunction in alzheimer’s disease.
  20. Trends in neurosciences, 46(2):124–136, 2023.
  21. Adel Javanmard and Andrea Montanari. Model selection for high-dimensional regression under the generalized irrepresentability condition. Advances in Neural Information Processing Systems, 26, 2013.
  22. Michael Komodromos, Marina Evangelou, Sarah Filippi, and Kolyan Ray. Group spike and slab variational Bayes. arXiv preprint arXiv:2309.10378, 2023.
  23. Wei-Ting Lai and Ray-Bing Chen. A review of Bayesian group selection approaches for linear regression models. Wiley Interdisciplinary Reviews: Computational Statistics, 13
  24. (4):e1513, 2021.
  25. Yi Lin and Hao Helen Zhang. Component selection and smoothing in smoothing in multivariate nonparametric regression. The Annals of Statistics, 38(5):2272–2297, 2006.
  26. Arjun V Masurkar. Towards a circuit-level understanding of hippocampal ca1 dysfunction in alzheimer’s disease across anatomical axes. Journal of Alzheimer’s disease & Parkinsonism, 8(1):412, 2018.
  27. Lukas Meier, Sara van de Geer, and Peter B¨uhlmann. High-dimensional additive modeling.
  28. The Annals of Statistics, 37(6(B)):3779–3821, 2009. Yuval Nardi and Alessandro Rinaldo.
  29. On the asymptotic properties of the group lasso estimator for linear models. Electronic Journal of Statistics, 9(2):605–633, 2008.
  30. Samhita Pal and Subhashis Ghosal. Bayesian high-dimensional linear regression with sparse projection-posterior. arXiv e-prints arXiv–2410, 2024.
  31. Trevor Park and George Casella. The Bayesian lasso. Journal of the American Statistical Association, 103(482):681–686, 2008.
  32. Sudhir Raman, Thomas J Fuchs, Peter J Wild, Edgar Dahl, and Volker Roth. The Bayesian group-lasso for analyzing contingency tables. In Proceedings of the 26th Annual International Conference on Machine Learning, pages 881–888, 2009.
  33. Y Lakshmisha Rao, B Ganaraja, BV Murlimanju, Teresa Joy, Ashwin Krishnamurthy, and
  34. Amit Agrawal. Hippocampus and its involvement in alzheimer’s disease: a review. 3 Biotech, 12(2):55, 2022.
  35. Pradeep Ravikumar, John Lafferty, Han Liu, and Larry Wasserman. Sparse additive models.
  36. Journal of the Royal Statistical Society Series B: Statistical Methodology, 71(5):1009–1030, 2009.
  37. Niranjan Subrahmanya and Yung C Shin. A variational Bayesian framework for group feature selection. International Journal of Machine Learning and Cybernetics, 4:609–619, 2013.
  38. Tingni Sun and Cun-Hui Zhang. Scaled sparse linear regression. Biometrika, 99(4):879–898, 2012.
  39. Sara van de Geer, Peter B¨uhlmann, Ya’acov Ritov, and Ruben Dezeure. On asymptotically optimal confidence regions and tests for high-dimensional models. The Annals of Statistics, 42(3):1166–1202, 2014.
  40. David L Wallace. Bounds on normal approximations to student’s and the chi-square distributions. The Annals of Mathematical Statistics, pages 1121–1130, 1959.
  41. Hansheng Wang and Chenlei Leng. A note on adaptive group lasso. Computational Statistics
  42. & Data Analysis, 52(12):5277–5286, 2008.
  43. Kang Wang and Subhashis Ghosal. Coverage of credible intervals in Bayesian multivariate isotonic regression. The Annals of Statistics, 51(3):1376–1400, 2023.
  44. Fengrong Wei and Jian Huang. Consistent group selection in high-dimensional linear regression. Bernoulli, 16(4):1369, 2010.
  45. Xiaofan Xu and Malay Ghosh. Bayesian variable selection and estimation for group lasso. Bayesian Analysis, 10(4):909–936, 2015.
  46. Zemei Xu, Daniel F Schmidt, Enes Makalic, Guoqi Qian, and John L Hopper. Bayesian grouped horseshoe regression with application to additive models. In AI 2016: Advances in Artificial Intelligence: 29th Australasian Joint Conference, Hobart, TAS, Australia, December 5-8, 2016, Proceedings 29, pages 229–240. Springer, 2016. Xinming Yang and Naveen N Narisetty.
  47. Consistent group selection with Bayesian high dimensional modeling. Bayesian Analysis, 15(3):909–935, 2020.
  48. Ming Yuan and Yi Lin. Model selection and estimation in regression with grouped variables.
  49. Journal of the Royal Statistical Society Series B: Statistical Methodology, 68(1):49–67, 2006.
  50. Lingmin Zeng and Jun Xie. Group variable selection via scad-l 2. Statistics, 48(1):49–66, 2014.
  51. Cun-Hui Zhang. Nearly unbiased variable selection under minimax concave penalty. Annals of Statistics, 38(2):894–942, 2010. Cun-Hui Zhang and Jian Huang.
  52. The sparsity and bias of the lasso selection in highdimensional linear regression. The Annals of Statistics, 36(4):1567–1594, 2008.
  53. Peng Zhao and Bin Yu. On model selection consistency of lasso. The Journal of Machine Learning Research, 7:2541–2563, 2006.
  54. Peng Zhao, Guilherme Rocha, and Bin Yu. The composite absolute penalties family for grouped and hierarchical variable selection. The Annals of Statistics, 37(6A):3468–3497, 2009.

Acknowledgments

Data collection and sharing for the Alzheimer’s Disease Neuroimaging Initiative (ADNI)

is funded by the National Institute on Aging (National Institutes of Health Grant U19

AG024904). The grantee organization is the Northern California Institute for Research and

Education.

8.