Abstract

We develop concentration inequalities for the l∞norm of vector lin

ear processes with sub-Weibull, mixingale innovations. This inequality is used

to obtain a concentration bound for the maximum entrywise norm of the lag-h

autocovariance matrix of linear processes. We apply these inequalities to sparse

estimation of large-dimensional VAR(p) systems and heterocedasticity and autocorrelation consistent (HAC) high-dimensional covariance estimation.

Information

Preprint No.SS-2023-0364
Manuscript IDSS-2023-0364
Complete AuthorsEduardo Fonseca Mendes, Fellipe Lopes Lima Leite
Corresponding AuthorsEduardo Fonseca Mendes
Emailseduardo.mendes@fgv.br

References

  1. Adamczak, R. (2015). A note on the Hanson-Wright inequality for random vectors with dependencies. Electronic Communications in Probability 20, 1 – 13.
  2. Adamek, R., S. Smeekes, and I. Wilms (2023a). Lasso inference for high-dimensional time series.
  3. Journal of Econometrics 235(2), 1114–1143.
  4. Adamek, R., S. Smeekes, and I. Wilms (2023b). Sparse high-dimensional vector autoregressive bootstrap. arXiv preprint, arXiv:2302.01233.
  5. Alquier, P. and P. Doukhan (2011). Sparsity considerations for dependent variables. Electronic Journal of Statistics 5, 750–774.
  6. Andrews, D. W. (1991). Heteroskedasticity and autocorrelation consistent covariance matrix estimation. Econometrica: Journal of the Econometric Society 59(3), 817–858.
  7. Babii, A., E. Ghysels, and J. Striaukas (2022). High-dimensional grander causality tests with an application to vix and news. Journal of Financial Econometrics 22(3), 605–635.
  8. Basu, S. and D. S. Matteson (2021). A survey of estimation methods for sparse high-dimensional time series models. arXiv preprint, arXiv:2107.14754.
  9. Basu, S. and G. Michailidis (2015). Regularized estimation in sparse high-dimensional time series models. The Annals of Statistics 43, 1535–1567.
  10. Beveridge, S. and C. R. Nelson (1981). A new approach to decomposition of economic time series into permanent and transitory components with particular attention to measurement of the ‘business cycle’. Journal of Monetary economics 7(2), 151–174.
  11. Bong, H. and A. K. Kuchibhotla (2023). Tight concentration inequality for sub-weibull random variables with generalized bernstien orlicz norm. arXiv preprint, arXiv:2302.03850.
  12. Bours, M. and A. Steland (2021). Large-sample approximations and change testing for highdimensional covariance matrices of multivariate linear time series and factor models. Scandinavian Journal of Statistics 48(2), 610–654.
  13. Brockwell, P. J. and R. A. Davis (2009). Time series: theory and methods. New York: Springer science & business media.
  14. Chen, L. and W. B. Wu (2018). Concentration inequalities for empirical processes of linear time series. J. Mach. Learn. Res. 18(231), 1–46.
  15. Dedecker, J., P. Doukhan, G. Lang, J. Le´on, S. Louhichi, and C. Prieur (2007). Weak dependence with examples and applications. Springer.
  16. Dedecker, J., F. Merlev`ede, and M. Peligrad (2011). Invariance principles for linear processes with application to isotonic regression. Bernoulli 17(1), 88–113.
  17. Doukhan, P. and M. H. Neumann (2007). Probability and moment inequalities for sums of weakly dependent random variables, with applications. Stochastic Processes and their Applications 117(7), 878–903.
  18. Fan, J., R. Masini, and M. C. Medeiros (2023). Bridging factor and sparse models. Annals of Statistics 51(4), 1692–1717.
  19. Fan, X., I. Grama, and Q. Liu (2012). Hoeffding’s inequality for supermartingales. Stochastic Processes and their Applications 122(10), 3545–3559.
  20. Fan, X., I. Grama, Q. Liu, et al. (2012). Large deviation exponential inequalities for supermartingales. Electronic Communications in Probability 17, 1–8.
  21. G¨otze, F., H. Sambale, and A. Sinulis (2021). Concentration inequalities for polynomials in α-sub-exponential random variables. Electronic Journal of Probability 26, 1–22.
  22. Hall, P. and C. C. Heyde (1980). Martingale limit theory and its application. New York: Academic press.
  23. Han, F., H. Lu, and H. Liu (2015). A direct estimation of high dimensional stationary vector autoregressions. The Journal of Machine Learning Research 16(1), 3115–3150.
  24. Hang, H. and I. Steinwart (2017). A bernstein-type inequality for some mixing processes and dynamical systems with an application to learning. Annals of Statistics 45(2), 708–743.
  25. Hansen, B. E. (1992). Consistent covariance matrix estimation for dependent heterogeneous processes. Econometrica: Journal of the Econometric Society 60(4), 967–972.
  26. Jiang, W. (2009). On uniform deviations of general empirical risks with unboundedness, dependence and high dimensionality. Journal of Machine Learning Research 10, 977–996.
  27. Kock, A. and L. Callot (2015). Oracle inequalities for high dimensional vector autoregressions. Journal of Econometrics 186, 325–344.
  28. Krampe, J., J.-P. Kreiss, and E. Paparoditis (2021). Bootstrap based inference for sparse high-dimensional time series models. Bernoulli 27(3), 1441–1466.
  29. Kuchibhotla, A. K. and A. Chakrabortty (2022). Moving beyond sub-Gaussianity in highdimensional statistics: applications in covariance estimation and linear regression. Information and Inference: A Journal of the IMA 11(4), 1398–1456.
  30. Lesigne, E. and D. Voln´y (2001). Large deviations for martingales. Stochastic Process and their Applications 96, 143 – 159.
  31. Li, J. and Z. Liao (2020). Uniform nonparametric inference for time series. Journal of Econometrics 219(1), 38–51.
  32. L¨utkepohl, H. (2006). New Introduction to Multiple Time Series Analysis. Berlin: SpringerVerlag.
  33. Marton, K. (1998). Measure concentration for a class of random processes. Probability Theory and Related Fields 110, 427–439.
  34. Masini, R. P., M. C. Medeiros, and E. F. Mendes (2022). Regularized estimation of highdimensional vector autoregressions with weakly dependent innovations. Journal of Time Series Analysis 43, 532–557.
  35. Masini, R. P., M. C. Medeiros, and E. F. Mendes (2023). Machine learning advances for time series forecasting. Journal of Economic Surveys 37(1), 76–111.
  36. Merlev`ede, F., M. Peligrad, and E. Rio (2009). Bernstein inequality and moderate deviations under strong mixing conditions. In C. Houdr´e, V. Koltchinskii, D. Mason, and M. Peligrad (Eds.), High Dimensional Probability V: The Luminy Volume, pp. 273–292. Institute of Mathematical Statistics.
  37. Mohri, M. and A. Rostamizadeh (2010). Stability bounds for stationary ϕ-mixing and β-mixing processes. Journal of Machine Learning Research 11(2), 789–814.
  38. Newey, W. K. and K. D. West (1987). A simple, positive semi-definite, heteroskedasticity and autocorrelation consistent covariance matrix. Econometrica 55(3), 703–708.
  39. Phillips, P. C. and V. Solo (1992). Asymptotics for linear processes. The Annals of Statistics 20(2), 971–1001.
  40. Politis, D. N. (2011). Higher-order accurate, positive semidefinite estimation of large-sample covariance and spectral density matrices. Econometric Theory 27(4), 703–744.
  41. Reuvers, H. and E. Wijler (2024). Sparse generalized yule–walker estimation for large spatiotemporal autoregressions with an application to no2 satellite data. Journal of Econometrics 239(1), Article:105520.
  42. Tao, T. and V. Vu (2013). Random matrices: Sharp concentration of eigenvalues. Random Matrices: Theory and Applications 2(03), Article:1350007.
  43. Tsay, R. S. (2013). Multivariate time series analysis: with R and financial applications. Hoboken, New Jersey: John Wiley & Sons.
  44. van der Vaart, A. and J. Wellner (1996). Weak Convergence and Empirical Processes: With Applications to Statistics. Springer.
  45. Vershynin, R. (2018). High-dimensional probability: An introduction with applications in data science, Volume 47. Cambridge University Press.
  46. Vladimirova, M., S. Girard, H. Nguyen, and J. Arbel (2020). Sub-weibull distributions: Generalizing sub-gaussian and sub-exponential properties to heavier tailed distributions. Stat 9(1), Article:e318.
  47. Wainwright, M. J. (2019). High-dimensional statistics: A non-asymptotic viewpoint, Volume 48. Cambridge university press.
  48. Wang, D. and R. S. Tsay (2023). Rate-optimal robust estimation of high-dimensional vector autoregressive models. Annals of Statistics 51(2), 846–877.
  49. Wang, Q., Y.-X. Lin, and C. M. Gulati (2001). Asymptotics for moving average processes with dependent innovations. Statistics & probability letters 54(4), 347–356.
  50. Wilms, I., S. Basu, J. Bien, and D. S. Matteson (2021). Sparse identification and estimation of large-scale vector autoregressive moving averages. Journal of the American Statistical Association 118(541), 571–582.
  51. Wong, K. C., Z. Li, and A. Tewari (2020). Lasso guarantees for β-mixing heavy-tailed time series. The Annals of Statistics 48(2), 1124–1142.
  52. Wu, W. B. (2005). Nonlinear system theory: Another look at dependence. Proceedings of the National Academy of Sciences 102(40), 14150–14154.
  53. Wu, W. B. and W. Min (2005). On linear processes with dependent innovations. Stochastic Processes and Their Applications 115(6), 939–958.
  54. Xiao, H. and W. B. Wu (2012). Covariance matrix estimation for stationary time series. The Annals of Statistics 40(1), 466–493.
  55. Yu, B. (1994, January).
  56. Rates of convergence for empirical processes of stationary mixing sequences. The Annals of Probability 22(1), 94–116.
  57. Zhang, D. and W. B. Wu (2017). Gaussian approximation for high-dimensional time series. The Annals of Statistics 45(5), 1895–1919.
  58. Zhang, H. and S. Chen (2021). Concentration inequalities for statistical inference. Communications in Mathematical Research 37(1), 1–85.
  59. Zhang, H. and H. Wei (2022). Sharper sub-weibull concentrations. Mathematics 10(13), Article:2252. Getulio Vargas Foundation, S˜ao Paulo School of Economics

Acknowledgments

We thank Professors Marcelo Fernandes and Luiz Max Carvalho, and two

anonymous referees for their insightful comments. The first author acknowledge financial support from the S˜ao Paulo Research Foundation (FAPESP)

(Grant No. 2023/01728-0).

Supplementary Materials

This supplement provides a more comprehensive analysis of the innovation

process and its characteristics and also includes proofs of all the results in

the paper. We derive concentration bounds that are used in the proof of

the triplex inequality in Section 2. Then, we present the results of Sections

3, 4 and 5 of the main paper, along with their respective proofs.

Supplementary materials are available for download.