Abstract

In high-dimensional data analysis, most sure independence screening (SIS) procedures are

significantly affected by both misspecification and missing data, making the results sensitive to the

loss of predictive accuracy. On the other hand, classical model averaging methods are typically limited

to well-specified structures or imposed restrictive constraints on candidates. To address the gaps, this

paper focuses on the conditional quantile estimation in conjunction with inverse probability weighting,

the purposes of which are mainly threefold. Firstly, we study the SIS properties under misspecified

quantile models. Secondly, we propose an adaptive model averaging algorithm for complex clusters.

Thirdly, we develop a robust improvement strategy to enhance asymptotic efficiency with respect to

high-dimensional ignorable mechanism. Theoretical properties of the averaging estimator are investigated, including its finite sample performance, the equivalence between adaptation and asymptotic

optimality, as well as the consistency of weights. Numerical simulations illustrate the method’s ability to efficiently identify the correct specification and maintain resilience against outliers in response

probabilities. The real-data example is analyzed to validate our method.

Information

Preprint No.SS-2024-0351
Manuscript IDSS-2024-0351
Complete AuthorsWei Xiong, Dianliang Deng, Wanying Zhang, Dehui Wang
Corresponding AuthorsDehui Wang
Emailswangdh@jlu.edu.cn

References

  1. Angrist, J., Chernozhukov, V., Fern´andez-Val, I. (2006). Quantile regression under misspecification, with an application to the U.S. wage structure. Econometrica 74, 539–563.
  2. Ando, T., Li, K. C. (2017). A weight-relaxed model averaging approach for high-dimensional generalized linear models. Annals of Statistics 45, 2654–2679.
  3. Austin, P. C., Stuart, E. A. (2017). The performance of inverse probability of treatment weighting and full matching on the propensity score in the presence of model misspecification when estimating the effect of treatment on survival outcomes. Statistical Methods in Medical Research 2017, 1654–1670.
  4. B¨uhlmann, P., van de Geer, S. (2011). Statistics for high-dimensional data: methods, theory and applications. Berlin: Springer-Verlag.
  5. Busso, M., DiNardo, J., McCrary, J. (2014). New evidence on the finite sample properties of propensity score reweighting and matching estimators. Review of Economics and Statistics 96, 885–897.
  6. Chen, Z., Liao, J., Xu, W., Yang, Y. (2023). Multifold cross-validation model averaging for generalized additive partial linear models. Journal of Computational and Graphical Statistics 32, 1649–1659.
  7. Chen, X., Wan A. T. K., Zhou, Y. (2015). Efficient quantile regression analysis With missing observations. Journal of the American Statistical Association 110, 723–741.
  8. Cook, R. D., Forzani, L. (2009). Likelihood-based sufficient dimension reduction. Journal of the American Statistical Association 104, 197–208.
  9. Crump, R. K., Imbens, G. W., Mitnik, O. A., Hotz, V. J. (2009). Dealing with limited overlap in estimation of average treatment effects. Biometrika 96, 187–199.
  10. Deng, J., Yang, X., Wang, Q. (2022). Surrogate space based dimension reduction for nonignorable nonresponse. Computational Statistics and Data Analysis 168, 107374.
  11. Fan, J., Lv, J. (2008). Sure independence screening for ultra-high dimensional feature space. Journal of the Royal Statistical Society: Series B 70, 849–911.
  12. Fan, J., Peng, H. (2003). Non-concave penalized likelihood with a diverging number of parameters. The Annals of Statistics 32, 928–961.
  13. Fan, J., Song, R. (2010). Sure independence screening in generalized linear models with NP-dimensionality. Annals of Statistics 38, 3567–3604.
  14. Fang, F., Yuan, C., Tian, W. (2023). An asymptotic theory for least squares model averaging with nested models. Econometric Theory 39, 412–441.
  15. Gu, Y., Zou, H. (2019). Aggregated expectile regression by exponential weighting. Statistica Sinica 29, 671–692.
  16. Hansen, B. E. (2007). Least squares model averaging. Econometrica 75, 1175–1189.
  17. He, B., Ma, S., Zhang, X., Zhu, L. X. (2023). Rank-based greedy model averaging for high-dimensional survival data. Journal of the American Statistical Association 118, 2658–2670.
  18. He, X., Wang, L., Hong, H. G. (2013). Quantile-adaptive model-free variable screening for high-dimensional heterogeneous data. Annals of Statistics 41, 342–369.
  19. Hoeting, J. A., Madigan, D., Raftery, A. E., Volinsky, C. T. (1999). Bayesian model averaging: a tutorial. Statistical Science 14, 382–401.
  20. Jiang, X., Liang, Y., Wang, H. (2024). Screen then select: a strategy for correlated predictors in high-dimensional quantile regression. Statistics and Computing 34, 112.
  21. Knight, K. (1998). Limiting distributions for l1 regression estimators under general conditions. Annals of Statistics 26, 755–770.
  22. Koenker, R. (2005). Quantile regression. New York: Cambridge University Press.
  23. Kong, Y., Li, Y., Zerom, D. (2019). Screening and selection for quantile regression using an alternative measure of variable importance. Journal of Multivariate Analysis 173, 435–455.
  24. Li, W., Gu, Y., Liu, L. (2020). Demystifying a class of multiply robust estimators. Biometrika 107, 919–933.
  25. Li, R., Zhong, W., Zhu, L. (2012). Feature screening via distance correlation learning. Journal of the American Statistical Association 107, 1129–1139.
  26. Little, R. J., Rubin, D. B. (2002). Statistical analysis with missing data. Wiley: New York.
  27. Lu, X., Su, L. (2015). Jackknife model averaging for quantile regressions. Journal of Econometrics 188, 40–58.
  28. Ma, X., Wang, J. (2020). Robust inference using inverse probability weighting. Journal of the American Statistical Association 115, 1851–1860.
  29. Ma, Y., Zhu, L. (2012). A semiparametric approach to dimension reduction. Journal of the American Statistical Association 107, 168–179.
  30. Mai, Q., Zou, H. (2013). The kolmogorov filter for variable screening in high-dimensional binary classification. Biometrika 100, 229–234.
  31. Shan, K., Yang, Y. (2009). Combining regression quantile estimators. Statistica Sinica 19, 1171–1191.
  32. Vershynin, R. (2010). Introduction to the non-asymptotic analysis of random matrices. arXiv preprint arXiv:1011.3027v7.
  33. Wang, B., Liang, H. (2023). Quantile regression of ultra-high dimensional partially linear varying-coefficient model with missing observations. Acta Mathematica Sinica: English Series 39, 1701–1726.
  34. Wang, L., Wu, Y., Li, R. (2012). Quantile regression for analyzing heterogeneity in ultra-high dimension. Journal of the American Statistical Association 107, 214–222.
  35. Wang, M., You, K., Zhu, L., Zou, G. (2024). Robust model averaging approach by mallows-type criterion. Biometrics 80, ujae128.
  36. Wang, M., Zhang, X., Wan, A. T. K., You, K., Zou, G. (2023). Jackknife model averaging for high-dimensional quantile regression. Biometrics 79, 178–189.
  37. White, H. (1982). Maximum likelihood estimation of misspecified models. Econometrica 50, 1–25.
  38. Wu, Y., Yin, G. (2015). Conditional quantile screening in ultrahigh-dimensional heterogeneous data. Biometrika 102, 65–76.
  39. Xie, J., Yan, X., Tang, N. (2021). A model-averaging method for high-dimensional regression with missing responses at random. Statistica Sinica 31, 1005–1026.
  40. Xiong, W., Deng, D., Wang, D. (2025). Semiparametric model averaging for high-dimensional quantile regression with nonignorable nonresponse. arXiv preprint arXiv:2509.00464.
  41. Yang, Y. (2004). Combining Forecasting procedures: some theoretical results. Econometric Theory 20, 176–222.
  42. Ye, C., Yang, Y., Yang, Y. (2018). Sparsity oriented importance learning for high-dimensional linear regression. Journal of the American Statistical Association 113, 1797–1812.
  43. Yu, D., Zhang, X., Liang, H. (2025). Unified optimal model averaging with a general loss function based on crossvalidation. Journal of the American Statistical Association, 1–12.
  44. Zeng, J., Hu, G., Cheng, W. (2024). A Mallows-type model averaging estimator for ridge regression with randomly right censored data. Statistics and Computing 34, 159.
  45. Zhang, X., Liu C. A. (2019). Inference after model averaging in linear regression models. Econometric Theory 35, 816–841.
  46. Zhang, X., Liu, C. A. (2023). Model averaging prediction by K-fold cross-validation. Journal of Econometrics 235, 280–301.
  47. Zhang, X., Zou, G., Liang, H., Carroll, R. J. (2020). Parsimonious model averaging with a diverging number of parameters. Journal of the American Statistical Association 115, 972–984.
  48. Zhao, P., Wang, L., Shao, J. (2020). Sufficient dimension reduction and instrument search for data with nonignorable nonresponse. Bernoulli 2, 930–945. Wei Xiong

Acknowledgments

The authors would like to thank the co-editor and anonymous referees for their constructive comments, which substantially improved the earlier version of the paper.

We also

appreciate Prof. Xinyu Zhang for suggestions on model averaging theory. Xiong’s work is

supported by National Natural Science Foundation of China (No.12401352), Postdoctoral

Fellowship Program of CPSF (No.GZC20231022), and China Postdoctoral Science Foundation (No.2025T180847). Deng’s work is supported by Natural Sciences and Engineering

Research Council of Canada (NSERC). Wang’s work is supported by National Natural Science Foundation of China (No.12271231, 12001229). The usual disclaimer applies.

Supplementary Materials

also show that high prediction precision for ˆπ(x) may not lead to

improved performance for (1.1). Note that the asymptotic optimality of (3.6) is not affected

Supplementary materials are available for download.