Abstract

We propose an estimator for precision matrices with the structure of Banded Kronecker

Sparse forms (BKS). BKS takes advantage of the special feature of a precision matrix, which has the

form of the kronecker product of an adaptively banded matrix and a sparse matrix, both are positive

definite. Such precision matrix arises frequently in practice in finance data, medical data and time

series data. We achieve the adaptive bandedness via a specially designed penalty, and enforce

the sparsity via lasso. We apply a computationally efficient procedure named Alternative Convex

Search (ACS) algorithm to implement BKS. We establish the computational convergence and

show the statistical guarantee through establishing the asymptotic rate. Our extensive simulation

studies indicate the superior finite sample performance of BKS in comparison to existing methods.

Additionally, we apply BKS to EEG and ADHD datasets, wherein it outperforms other methods

in capturing the banding sparsity characteristics of the precision matrix.

Information

Preprint No.SS-2023-0131
Manuscript IDSS-2023-0131
Complete AuthorsChunhui Liang, Wenqing Ma, Yanyuan Ma
Corresponding AuthorsWenqing Ma
Emailswenqingma@cnu.edu.cn

References

  1. Aston, J. A. D., D. Pigoli, and S. Tavakoli (2017). Tests for separability in nonparametric covariance operators of random surfaces. The Annals of Statistics 45(4), 1431–1461.
  2. Banerjee, O., L. El Ghaoui, and A. d’Aspremont (2008). Model selection through sparse maximum likelihood estimation for multivariate gaussian or binary data. The Journal of Machine Learning Research 9(3), 485–516.
  3. Bickel, P. J. and E. Levina (2008a). Covariance regularization by thresholding. Annals of Statistics 36, 2577–2604.
  4. Bickel, P. J. and E. Levina (2008b). Regularized estimation of large covariance matrices. Annals of Statistics 36(1), 199–227.
  5. Bien, J., F. Bunea, and L. Xiao (2016). Convex banding of the covariance matrix. Journal of the American Statistical Association 111(514), 834–845.
  6. Boyd, S., N. Parikh, E. Chu, B. Peleato, and J. Eckstein (2011). Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends in Machine learning 3(1), 1–122.
  7. Cai, T. T., C. H. Zhang, and H. H. Zhou (2010). Optimal rates of convergence for covariance matrix estimation. Annals of Statistics 38(1), 2118–2144.
  8. Dai, D., C. Hao, S. Jin, and Y. Liang (2023). Regularized estimation of kronecker structured covariance matrix using modified cholesky decomposition. Journal of Statistical Computation and Simulation 95(5), 1–26.
  9. Friedman, J., T. Hastie, and R. Tibshirani (2008). Sparse inverse covariance estimation with the graphical lasso. Biostatistics 9(3), 432–441.
  10. Furrer, R. and T. Bengtsson (2007). Estimation of high-dimensional prior and posterior covariance matrices in kalman filter variants. Journal of Multivariate Analysis 98(1), 227–255.
  11. Gorski, J., F. Pfeuffer, and K. Klamroth (2007). Biconvex sets and optimization with biconvex functions: a survey and extensions. Mathematical Methods of Operations Research 66, 373408.
  12. Greenewald, K. and A. O. Hero (2015). Robust kronecker product pca for spatio-temporal covariance estimation. IEEE Transactions on Signal Processing 63(23), 6368–6378.
  13. Jenatton, R., J.-Y. Audibert, and F. Bach (2011). Structured variable selection with sparsity-inducing norms. Journal of Machine Learning Research 12, 2777–2824.
  14. Kim, C. and D. L. Zimmerman (2012). Unconstrained models for the covariance structure of multivariate longitudinal data. Journal of Multivariate Analysis 107, 104–118.
  15. Lee, K., H. Cho, M. Kwak, and E. Jang (2020). Estimation of covariance matrix of multivariate longitudinal data using modified choleksky and hypersphere decompositions. Journal of Multivariate Analysis 76(1), 75–86.
  16. Leng, C. and G. Pan (2018). Covariance estimation via sparse kronecker structures. Bernoulli 24(4B), 3833–3863.
  17. Leng, C. and C. Tang (2012). Sparse matrix graphical models. Journal of the American Statistical Association 107(499), 1187–1200.
  18. Qian, F., Y. Chen, and W. Zhang (2020). Regularized estimation of precision matrix for high-dimensional multivariate longitudinal data. Journal of Multivariate Analysis 176, 104580.
  19. Qian, F., W. Zhang, and Y. Chen (2021). Adaptive banding covariance estimation for high-dimensional multivariate longitudinal data. Canadian Journal of Statistics 49(3), 906–938.
  20. Rothman, A., E. Levina, and J. Zhu (2008). Sparse permutation invariant covariance estimation. Electronic Journal of Statistics 2, 494–515.
  21. Tsiligkaridis, T. and A. O. Hero (2013). Covariance estimation in high dimensions via kronecker product expansions. IEEE Transactions on Signal Processing 61(21), 5347–5360.
  22. Yu, G. and J. Bien (2017). Learning local dependence in ordered data. Journal of Machine Learning Research 18(42), 1–60.
  23. Yuan, M. and Y. Lin (2006). Model selection and estimation in regression with grouped variables. Journal of the Royal Statistical Society: Statistical Methodology Series B 68, 49–67.
  24. Yuan, M. and Y. Lin (2007). Model selection and estimation in the gaussian graphical model. Biometrika 94(1), 19–35.
  25. Zhang, X. L., H. Begleiter, B. Porjesz, W. Wang, and A. Litke (1995). Event related potentials during object recognition tasks. Brain research bulletin 6, 531–538.
  26. Zhang, Y., W. Shen, and K. Dehan (2023). Covariance estimation for matrix-valued data. Journal of the American Statistical Association 118(554), 2620–2631. KLAS and Department of Mathematics and Statistics, Northeast Normal University,5268 Renmin Street, Changchun, China

Acknowledgments

We are very grateful to the Editor,Associate Editor and referees, as well as our financial

sponsors for their insightful comments and suggestions that have improved the manuscript

significantly. This work was supported by the Key Program of the National Natural

Science Foundation of China [grant number 12431009].

Supplementary Materials

The supplementary materials provides the proofs of the lemmas, Theorem 1, Theorem 2,

Theorem 3 and Theorem 4, and these figures in simulation and real data studies.

Supplementary materials are available for download.