Abstract

Distributed learning enables scalable data analysis by reducing stor

age and computational burdens, although the communication of local outputs

in divide-and-conquer (DC) implementations may introduce privacy risks. We

study distributed estimation and inference for the single-index model (SIM),

a flexible semiparametric framework with an interpretable low-dimensional index structure. We propose a DC estimator for the SIM and establish conver-

gence rates and error bounds for the nonparametric component, together with

asymptotic normality of the index parameter estimator. Under mild conditions,

the aggregated estimator attains the optimal nonparametric convergence rate

(

N −p/(2p+1))

.

To provide rigorous privacy protection, we further develop

OP

differentially private (DP) algorithms for the SIM under the DC framework that

satisfy (ε, δ)-DP. We establish a high-probability bound on the average squared

gradient mapping, thereby characterizing how privacy noise and model complexity affect optimization accuracy.

Simulations and a real-data application

demonstrate accurate estimation and effective privacy protection.

Key words and phrases: Distributed learning; Differential privacy; Single-index model; Optimal convergence rate; Asymptotic properties 1

Information

Preprint No.SS-2025-0406
Manuscript IDSS-2025-0406
Complete AuthorsFode Zhang, Lingrui Wang, Hua Liang
Corresponding AuthorsFode Zhang
Emailsfredzh@swufe.edu.cn

References

  1. Avella-Medina, M. (2021). Privacy-preserving parametric inference: A case for robust statistics. Journal of the American Statistical Association 116(534), 969–983.
  2. Awan, J. and S. Vadhan (2023). Canonical noise distributions and private hypothesis tests. The Annals of Statistics 51(2), 547–572.
  3. Balabdaoui, F., C. Durot, and H. Jankowski (2019). Least squares estimation in the monotone single index model. Bernoulli 25(4B), 3276–3310.
  4. Bao, Y. and W. Xiong (2021). One-round communication efficient distributed m-estimation. In International Conference on Artificial Intelligence and Statistics, pp. 46–54. PMLR.
  5. Barrientos, A. F., A. R. Williams, J. Snoke, and C. M. Bowen (2024). A feasibility study of differentially private summary statistics and regression analyses with evaluations on administrative and survey data. Journal of the American Statistical Association 119(545), 52–65.
  6. Cai, T. T., Y. Wang, and L. Zhang (2021). The cost of privacy: Optimal rates of convergence for parameter estimation with differential privacy. The Annals of Statistics 49(5), 2825–2850.
  7. Chen, X., W. Jing, W. Liu, and Y. Zhang (2024). Distributed estimation and inference for semiparametric binary response models. The Annals of Statistics 52(3), 922–947.
  8. Chen, X. and M. Xie (2014). A split-and-conquer approach for analysis of extraordinarily large data. Statistica Sinica 24(4), 1655–1684.
  9. Cui, X., W. K. Härdle, and L. Zhu (2011). The efm approach for single-index models. The Annals of Statistics 39(3), 1658–1688.
  10. De Boor, C. (1978). A practical guide to splines, Volume 27. Springer New York.
  11. Dong, C., J. Gao, and D. Tjøstheim (2016). Estimation for single-index and partially linear single-index integrated models. The Annals of Statistics 44(1), 425–453.
  12. Dong, J., A. Roth, and W. J. Su (2022). Gaussian differential privacy. Journal of the Royal Statistical Society Series B: Statistical Methodology 84(1), 3–37.
  13. Dwork, C., F. McSherry, K. Nissim, and A. Smith (2006). Calibrating noise to sensitivity in private data analysis. In In TCC 2006, pp. 265–284. Springer.
  14. Fan, J., Y. Guo, and K. Wang (2023). Communication-efficient accurate statistical estimation. Journal of the American Statistical Association 118(542), 1000–1010.
  15. Gao, Y., W. Liu, H. Wang, X. Wang, Y. Yan, and R. Zhang (2022). A review of distributed statistical inference. Statistical Theory and Related Fields 6(2), 89–99.
  16. Gu, J. and S. X. Chen (2023). Distributed statistical inference under heterogeneity. Journal of Machine Learning Research 24(387), 1–57.
  17. Györfi, L. and M. Kroll (2025). On rate optimal private regression under local differential privacy. Statistica Sinica 35, 613–627.
  18. Han, D., M. Han, J. Huang, and Y. Lin (2023). Robust inference for high-dimensional single index models. Scandinavian Journal of Statistics 50(4), 1590–1615.
  19. Horowitz, J. L. (2009). Single index models. In Semiparametric and Nonparametric Methods in
  20. Econometrics, New York, NY, pp. 7–51. Springer Berlin Heidelberg.
  21. Huang, H., J. Shangguan, X. Li, and H. Liang (2020). High-dimensional single-index models with censored responses. Statistics in Medicine 39(21), 2695–2854.
  22. Jiang, R. and Y. Peng (2023). A short note on fitting a single-index model with massive data. Statistical Theory and Related Fields 7(1), 49–60.
  23. Jordan, M. I., J. D. Lee, and Y. Yang (2019). Communication-efficient distributed statistical inference. Journal of the American Statistical Association 114(526), 668–681.
  24. Kuchibhotla, A. K. and R. K. Patra (2020). Efficient estimation in single index models through smoothing splines. Bernoulli 26(2), 1587–1618.
  25. Li, K.-C. (1991). Sliced inverse regression for dimension reduction. Journal of the American Statistical Association 86(414), 316–327.
  26. Lian, H., H. Liang, and R. J. Carroll (2014). Variance function partially linear single-index models. Journal of the Royal Statistical Society: Series B 77(1), 171–194.
  27. Tu, J., W. Liu, and X. Mao (2024). Distributed estimation on semi-supervised generalized linear model. Journal of Machine Learning Research 25, 1–41.
  28. Wang, D., M. Ye, and J. Xu (2017). Differentially private empirical risk minimization revisited: Faster and more general. Advances in Neural Information Processing Systems 30.
  29. Wang, L. and L. Yang (2009). Spline estimation of single-index models. Statistica Sinica 19(2), 765–783.
  30. Xia, Y. and W. Härdle (2006). Semi-parametric estimation of partially linear single-index models. Journal of Multivariate Analysis 97(5), 1162–1184.
  31. Yan, T. (2021). Directed networks with a differentially private bi-degree sequence. Statistica Sinica 31(4), 2031–2050.
  32. Yu, Y. and D. Ruppert (2002). Penalized spline estimation for partially linear single-index models. Journal of the American Statistical Association 97(460), 1042–1054.
  33. Zhang, Y., J. Duchi, and M. Wainwright (2015). Divide and conquer kernel ridge regression: A distributed algorithm with minimax optimal rates. The Journal of Machine Learning Research 16(1), 3299–3340. Lingrui Wang, Center of Statistical Research, School of Statistics and Data Science, Southwestern University of Finance and Economics, Chengdu 611130, China.

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China (Nos. 12071372, 12201395, 11901134), in part by the Sichuan

Science and Technology Program (No. 2024ZYD0135).

Supplementary Materials

The supplemental materials contain an appendix with proofs, technical

lemmas, complete experimental results, and additional experiments.

Supplementary materials are available for download.