Abstract

This paper explores high-dimensional time dependent regression models within a

transfer learning framework. Specifically, we develop an estimator for the regression parameters of the high-dimensional linear models in time series settings based on transfer learning,

and establish the convergence rate of the proposed estimator. Our results reveal that leveraging auxiliary data can substantially enhance the convergence rate of the proposed estimator

compared to the traditional single-task approaches. For statistical inference of the target

regression coefficients, we propose a novel debiased method based on transfer learning that

incorporates a banded estimator of the error autocovariance matrix and demonstrate its

asymptotic normality. To mitigate the risk of negative transfer, we develop a transferable

source detection algorithm that adapts to data dependence, guaranteeing correct selection of

auxiliary samples that are sufficiently similar to the target samples. By leveraging information

from multiple tasks, our method enhances both robustness and accuracy in the estimation

process, ultimately improving statistical inference performance. Numerical simulations and

a real-data experiment reveal significant improvements in estimation and inference accuracy

compared to both the single-task Lasso regression and the transfer learning methods for

independent data.

Key words and phrases: high-dimensional linear models; source detection; statistical inference; 1

Information

Preprint No.SS-2025-0335
Manuscript IDSS-2025-0335
Complete AuthorsZongqi Liu, Shengji Jia, Xiao Guo
Corresponding AuthorsXiao Guo
Emailsxiaoguo@ustc.edu.cn

References

  1. Adamek, R., Smeekes, S., and Wilms, I. (2023). Lasso inference for high-dimensional time series. Journal of Econometrics, 235:1114–1143.
  2. Bastani, H. (2021). Predicting with proxies: transfer learning in high dimension. Management Science, 67:2964– 2984.
  3. Basu, S. and Michailidis, G. (2015). Regularized estimation in sparse high-dimensional time series models. The Annals of Statistics, 43:1535–1567.
  4. Bickel, P. J. and Levina, E. (2008). Regularized estimation of large covariance matrices. The Annals of Statistics, 36:199–227.
  5. Chernozhukov, V., Karl H¨ardle, W., Huang, C., and Wang, W. (2021). Lasso-driven inference in time and space. The Annals of Statistics, 49:1702–1735.
  6. Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.
  7. Duan, J., Pelger, M., and Xiong, R. (2024). Target PCA: Transfer learning large dimensional panel data. Journal of Econometrics, 244:105–121.
  8. Fan, J., Liao, Y., and Mincheva, M. (2011). High dimensional covariance matrix estimation in approximate factor models. The Annals of Statistics, 39:3320–3356.
  9. Hajiramezanali, E., Zamani Dadaneh, S., Karbalayghareh, A., Zhou, M., and Qian, X. (2018). Bayesian multidomain learning for cancer subtype discovery from next-generation sequencing count data. Neural Information Processing Systems, 31:9133–9142.
  10. Hannan, E. J. and Deistler, M. (2012). The Statistical Theory of Linear Systems. SIAM, New York.
  11. He, B., Liu, H., Zhang, X., and Huang, J. (2024). Representation transfer learning for semiparametric regression. arXiv preprint arXiv:2406.13197.
  12. Javanmard, A. and Montanari, A. (2014). Confidence intervals and hypothesis testing for high-dimensional regression. Journal of Machine Learning Research, 15:2869–2909.
  13. Kolmogoroff, A. (1928). ¨Uber die Summen durch den Zufall bestimmter unabh¨angiger Gr¨oßen. Mathematische Annalen, 99:309–319.
  14. Li, D., Nguyen, H. L., and Zhang, H. R. (2023a). Identification of negative transfers in multitask learning using surrogate models. arXiv preprint arXiv:2303.14582.
  15. Li, S. (2020). Debiasing the debiased lasso with bootstrap. Electronic Journal of Statistics, 14:2298–2337.
  16. Li, S., Cai, T. T., and Li, H. (2022). Transfer learning for high-dimensional linear regression: prediction, estimation and minimax optimality. Journal of the Royal Statistical Society Series B: Statistical Methodology, 84:149–173.
  17. Li, S., Cai, T. T., and Li, H. (2023b). Transfer learning in large-scale gaussian graphical models with false discovery rate control. Journal of the American Statistical Association, 118:2171–2183.
  18. Li, S., Zhang, L., Cai, T. T., and Li, H. (2024). Estimation and inference for high-dimensional generalized linear models with knowledge transfer. Journal of the American Statistical Association, 119:1274–1285.
  19. Lin, Y., Zhu, Q., and Li, G. (2025). Improving time series estimation and prediction via transfer learning. arXiv preprint arXiv:2510.25236.
  20. Ma, M. and Safikhani, A. (2025). Transfer learning for high-dimensional reduced rank time series models. Proceedings of Machine Learning Research, 258:2926–2934.
  21. Ma, Y., Gong, W., and Mao, F. (2015). Transfer learning used to analyze the dynamic evolution of the dust aerosol. Journal of Quantitative Spectroscopy and Radiative Transfer, 153:119–130.
  22. Nguyen, T.-T. and Yoon, S. (2019). A novel approach to short-term stock price movement prediction using transfer learning. Applied Sciences, 9:4745.
  23. Pan, S. J. and Yang, Q. (2009). A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering, 22:1345–1359.
  24. Tian, Y. and Feng, Y. (2023). Transfer learning under high-dimensional generalized linear models. Journal of the American Statistical Association, 118:2684–2697.
  25. Torrey, L. and Shavlik, J. (2010). Transfer learning. In Handbook of Research on Machine Learning Applications and Trends: Algorithms, Methods, and Techniques, pages 242–264. IGI global, Hershey, Pennsylvania.
  26. Tripuraneni, N., Jordan, M., and Jin, C. (2020). On the theory of transfer learning: the importance of task diversity. Neural Information Processing Systems, 33:7852–7862.
  27. Van de Geer, S., B¨uhlmann, P., Ritov, Y., and Dezeure, R. (2014). On asymptotically optimal confidence regions and tests for high-dimensional models. The Annals of Statistics, 42:1166–1202.
  28. Weiss, K., Khoshgoftaar, T. M., and Wang, D. (2016). A survey of transfer learning. Journal of Big Data, 3:1–40.
  29. Wu, S., Zhang, H. R., and R´e, C. (2020). Understanding and improving information transfer in multi-task learning. arXiv preprint arXiv:2005.00944.
  30. Wu, W. B. (2005). Nonlinear system theory: another look at dependence. Proceedings of the National Academy of Sciences of the United States of America, 102:14150–14154.
  31. Wu, W. B. and Wu, Y. (2016). Performance bounds for parameter estimates of high-dimensional linear models with correlated errors. Electronic Journal of Statistics, 10:352–379.
  32. Xia, J., Chen, Y., and Guo, X. (2024). Inference for high-dimensional linear models with locally stationary error processes. Journal of Time Series Analysis, 45:78–102.
  33. Xiao, J., Hu, Y., Xiao, Y., Xu, L., and Wang, S. (2017). A hybrid transfer learning model for crude oil price forecasting. Statistics and Its Interface, 10:119–130.
  34. Yuan, P. and Guo, X. (2022). High-dimensional inference for linear model with correlated errors. Metrika, 85:21–52.
  35. Zhang, C. and Zhang, S. S. (2014). Confidence intervals for low dimensional parameters in high dimensional linear models. Journal of the Royal Statistical Society Series B: Statistical Methodology, 76:217–242.
  36. Zhang, D. and Wu, W. B. (2017). Gaussian approximation for high dimensional time series. The Annals of Statistics, 45:1895–1919.
  37. Zhao, L., Pan, S., Xiang, E., Zhong, E., Lu, Z., and Yang, Q. (2013). Active transfer learning for cross-system recommendation. Proceedings of the AAAI Conference on Artificial Intelligence, 27:1205–1211.
  38. Zhou, J., Yu, Q., Luo, C., and Zhang, J. (2023). Feature decomposition for reducing negative transfer: a novel multi-task learning method for recommender system (student abstract). Proceedings of the AAAI conference on Artificial Intelligence, 37:16390–16391.
  39. Zhu, Y. and Bradic, J. (2018). Significance testing in non-sparse high-dimensional linear models. Electronic Journal of Statistics, 12:3312–3364.
  40. Zhu, Y., Yu, W., and Li, X. (2025). A multi-objective transfer learning framework for time series forecasting with Concept Echo State Networks. Neural Networks, 186:107272. Zongqi Liu

Acknowledgments

We thank the editor, associate editor and two referees for their helpful comments and

suggestions. Jia’s research was partially supported by National Natural Science Foundation

of China, Grant 12501374, and Shanghai Natural Science Foundation, Grant 25ZR1402404.

Guo’s research was supported by the National Natural Science Foundation of China, Grant

12471267.

Supplementary Materials

The Supplementary Material includes detailed proofs of the main theorems and necessary

lemmas, additional numerical results, and comprehensive guidelines for tuning parameter

selection.

Supplementary materials are available for download.