Abstract

In this article, we introduce the mean independent component analysis for multivariate time series

to reduce the parameter space. In particular, we seek for a contemporaneous linear transformation that

detects univariate mean independent components so that each component can be modeled separately. The

mean independent component analysis is flexible in the sense that no parametric model or distributional

assumptions are made. We propose a unified framework to estimate the mean independent components from

a data with a fixed dimension or a diverging dimension. We estimate the mean independent components by

the martingale difference divergence so that the mean dependence across components and across time is

minimized. The approach is extended to the group mean independent component analysis by imposing

a group structure on the mean independent components. We further introduce a method to identify the

group structure when it is unknown. The consistency of both proposed methods is established. Extensive

simulations and a real data illustration for community mobility is provided to demonstrate the efficacy of

our method.

Key words and phrases: Conditional mean, Dimension reduction, High dimensional time series, Nonlinear dependence

Information

Preprint No.SS-2024-0181
Manuscript IDSS-2024-0181
Complete AuthorsChung Eun Lee, Zeda Li
Corresponding AuthorsChung Eun Lee
Emailschungeun.lee@baruch.cuny.edu

References

  1. Back, A. D. & Weigend, A. S. (1997), ‘A first application of independent component analysis to extracting structure from stock returns’, International journal of neural systems 8(04), 473–484.
  2. Bai, J. & Ng, S. (2002), ‘Determining the number of factors in approximate factor models’, Econometrica 70(1), 191–221.
  3. Belouchrani, A., Abed-Meraim, K., Cardoso, J.-F. & Moulines, E. (1997), ‘A blind source separation technique using second-order statistics’, IEEE Transactions on signal processing 45(2), 434–444.
  4. Box, G. E. & Tiao, G. C. (1977), ‘A canonical analysis of multiple time series’, Biometrika 64(2), 355–365.
  5. Calder´on-Ju´arez, M., Gonz´alez-G´omez, G. H., Echeverr´ıa, J. C. & Lerma, C. (2023), ‘Revisiting nonlinearity of heart rate variability in healthy aging’, Scientific Reports 13(1), 13185.
  6. Cardoso, J.-F. (1998), Multidimensional independent component analysis, in ‘Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP’98 (Cat. No. 98CH36181)’, Vol. 4, IEEE, pp. 1941–1944.
  7. Chang, J., Guo, B. & Yao, Q. (2015), ‘High dimensional stochastic regression with latent factors, endogeneity and nonlinearity’, Journal of Econometrics 189(2), 297–312.
  8. Chang, J., Guo, B. & Yao, Q. (2018), ‘Principal component analysis for second-order stationary vector time series’, The Annals of Statistics 46(5), 2094–2124.
  9. Chen, E. Y., Tsay, R. S. & Chen, R. (2020), ‘Constrained factor models for high-dimensional matrix-variate time series’, Journal of the American Statistical Association 115, 775–793.
  10. Chen, R., Yang, D. & Zhang, C.-H. (2021), ‘Factor models for high-dimensional tensor time series’, Journal of the American Statistical Association pp. 1–23.
  11. Dette, H. & Spreckelsen, I. (2004), ‘Some comments on specification tests in nonparametric absolutely regular processes’, Journal of Time Series Analysis 25(2), 159–172.
  12. Forni, M., Hallin, M., Lippi, M. & Reichlin, L. (2005), ‘The generalized dynamic factor model: one-sided estimation and forecasting’, Journal of the American statistical association 100(471), 830–840.
  13. Gruber, P., Gutch, H. W. & Theis, F. J. (2009), Hierarchical extraction of independent subspaces of unknown dimensions, in ‘International Conference on Independent Component Analysis and Signal Separation’, Springer, pp. 259–266.
  14. Han, Y., Chen, R., Zhang, C.-H. & Yao, Q. (2024), ‘Simultaneous decorrelation of matrix time series’, Journal of the American Statistical Association 119(546), 957–969.
  15. Hjellvik, V., Yao, Q. & Tjøstheim, D. (1996), Linearity testing using local polynomial approximation. discussion paper 60, Technical report, Sonderforschungsbereich 373, Humboldt-Universit¨at zu Berlin, Spandauerst. 1, 10178, Berlin.
  16. Hyvarinen, A., Karhunen, J. & Oja, E. (2001), ‘Independent component analysis [m]’, New York: A Wiley Interscience Publication pp. 175–182.
  17. Lam, C. & Yao, Q. (2012), ‘Factor modeling for high-dimensional time series: inference for the number of factors’, The Annals of Statistics pp. 694–726.
  18. Li, Z. (2023), ‘Robust conditional spectral analysis of replicated time series’, Statistics and Its Interface 16(1), 81–96.
  19. Lobato, I. N., Nankervis, J. C. & Savin, N. (2002), ‘Testing for zero autocorrelation in the presence of statistical dependence’, Econometric Theory 18(3), 730–743.
  20. Marwan, N., Donges, J. F., Donner, R. V. & Eroglu, D. (2021), ‘Nonlinear time series analysis of palaeoclimate proxy records’, Quaternary Science Reviews 274, 107245.
  21. Matteson, D. S. & Tsay, R. S. (2011), ‘Dynamic orthogonal components for multivariate time series’, Journal of the American Statistical Association 106(496), 1450–1463.
  22. Matteson, D. S. & Tsay, R. S. (2017), ‘Independent component analysis via distance covariance’, Journal of the American Statistical Association 112(518), 623–637.
  23. Miettinen, J., Illner, K., Nordhausen, K., Oja, H., Taskinen, S. & Theis, F. J. (2016), ‘Separation of uncorrelated stationary time series using autocovariance matrices’, Journal of Time Series Analysis 37(3), 337–354.
  24. Munch, S. B., Rogers, T. L., Symons, C. C., Anderson, D. & Pennekamp, F. (2023), ‘Constraining nonlinear time series modeling with the metabolic theory of ecology’, Proceedings of the National Academy of Sciences 120(12), e2211758120.
  25. Pan, J. & Yao, Q. (2008), ‘Modelling multiple time series via common factors’, Biometrika 95(2), 365–379.
  26. Park, T., Shao, X. & Yao, S. (2015), ‘Partial martingale difference correlation’, Electronic Journal of Statistics 109, 1492–1517.
  27. Pe˜na, D. & Box, G. E. (1987), ‘Identifying a simplifying structure in time series’, Journal of the American statistical Association 82(399), 836–843.
  28. Shao, X. & Wu, W. B. (2007), ‘Asymptotic spectral theory for nonlinear time series’.
  29. Shao, X. & Zhang, J. (2014), ‘Martingale difference correlation and its use in highdimensional variable screening’, Journal of the American Statistical Association 109, 1302–1318.
  30. Stewart, G. W. (1980), ‘The efficient generation of random orthogonal matrices with an application to condition estimators’, SIAM Journal on Numerical Analysis 17(3), 403–409.
  31. Stock, J. H. & Watson, M. W. (2002), ‘Forecasting using principal components from a large number of predictors’, Journal of the American statistical association 97(460), 1167–1179.
  32. St¨ogbauer, H., Kraskov, A., Astakhov, S. A. & Grassberger, P. (2004), ‘Least-dependentcomponent analysis based on mutual information’, Physical Review E 70(6), 066123.
  33. Sz´ekely, G. J., Rizzo, M. L. & Bakirov, N. K. (2007), ‘Measuring and testing dependence by correlation of distances’.
  34. Ter¨asvirta, T., Tjøstheim, D. & Granger, C. W. (2010), Modelling nonlinear economic time series, Oxford University Press.
  35. Tiao, G. C. & Tsay, R. S. (1989), ‘Model specification in multivariate time series’, Journal of the Royal Statistical Society: Series B (Methodological) 51(2), 157–195.
  36. Tong, L., Inouye, Y. & Liu, R.-W. (1992), ‘A finite-step global convergence algorithm for the parameter estimation of multichannel ma processes’, IEEE Transactions on signal processing 40(10), 2547–2558.
  37. Tsay, R. S. (1998), ‘Testing and modeling multivariate threshold models’, journal of the american statistical association 93(443), 1188–1202.
  38. Wang, D., Liu, X. & Chen, R. (2019), ‘Factor models for matrix-valued high-dimensional time series’, Journal of econometrics 208(1), 231–248.
  39. Wang, G., Zhu, K. & Shao, X. (2022), ‘Testing for the martingale difference hypothesis in multivariate time series models’, Journal of Business & Economic Statistics 40(3), 980– 994.
  40. Wen, Z. & Yin, W. (2013), ‘A feasible method for optimization with orthogonality constraints’, Mathematical Programming 142(1-2), 397–434.
  41. Zhen, Y. & Wang, J. (2023), ‘Non-negative tensor completion for dynamic counterfactual prediction on covid-19 pandemic’, Annals of Applied Statistics . Algorithm 1: Algorithm to estimate the group mean independent components and the group structure Data: Yt = (y1,t, . . . , yp,t)⊤ Result: bA = ( bA1, · · · , bAm), bXt = (bx1,t, · · · bxp,t)⊤, and (bp1, · · · , bp bm). Step 1: Begin the algorithm by setting m = p and p1 = · · · = pm = 1. Obtain an initial estimate bA(0) by minimizing bSh0(·) in (3.4) and estimate the components by bX(0) t = ( bA(0))⊤Yt. Step 2: Compute bM(i, j) defined in (4.11) for every two pairs of components and arrange bM(i, j) in the descending order. Step 3: Select r through the ratio-based estimator in (4.10) with presepcified c0 and collect ( bM1, · · · , bMbr), where bMk is the kth largest bM(i, j). Step 4: Based on the collected ( bM1, · · · , bMbr), create an undirected graph G = (V, E), where V = {1, 2, · · · , p} is the vertex set and E is the set of edges such that ei,j = ej,i = 1 if bM(i, j) ∈( bM1, · · · , bMbr) or ei,j = ej,i = 0 if bM(i, j) ̸∈( bM1, · · · , bMbr). Step 5: Based on the graph in Step 4, estimate the group structure, (bp1, · · · , bp bm). For instance, two components i and j belong to the same group if the vertices i and j are directly connected or indirectly connected, i.e., ei,j = 1 or there exists {v1, v2, · · · , vw} ⊂V such that ei,v1 = ev1,v2 = · · · = evw,j = 1. Step 6: Estimate bA(1) by minimizing bGh0(·) in (4.9) with (bp1, · · · , bp bm) obtained from Step 5. Permute bA(1) based on the estimated group structure and estimate the components by bX(1) t = ( bA(1))⊤Yt, where bA(1) is the permuted matrix. Step 7: Repeat Step 2 - Step 6 until the estimated group structure, (bp1, · · · , bp bm), does not change and ∥bA(i+1) −bA(i)∥F < ϵ, where bA(i+1) and bA(i) are the estimates of A after ith

Acknowledgments

The authors thank the Editor, the Associate Editor, and two referees for their constructive

comments and suggestions that led to substantial improvements. Dr. Lee’s research is supported by NSF grant DMS-2532852. Dr. Li’s research is supported by NSF grant DMS-

2418850.

Supplementary Materials

available online includes technical proofs of theoretical results and

state additional theorem with its proof, and reports additional simulations, real data application results, and figures.

Supplementary materials are available for download.