Abstract

Supervised learning typically assumes access to accurate ground truth labels, but in many real

world applications, label acquisition is costly, time-consuming, and often prone to expert disagreement and

annotation noise. We address this challenge in the context of functional imaging data by proposing the

annotated functional deep neural network (afDNN), a novel classification framework for noisy annotated

imaging data. By modeling images as random functions over a bounded spatial domain, we extract projection

scores and use them as inputs to a sparse deep neural network to jointly estimate the ground truth label

distribution and annotator-specific confusion matrices. Our method incorporates a regularized cross-entropy

loss to ensure identifiability of the annotator noise structure and requires no anchor labels or prior knowledge

of annotator reliability. We establish theoretical convergence guarantees for the proposed estimator. Extensive

simulations and application to chest X-ray and brain imaging datasets demonstrate the effectiveness and

robustness of the proposed method.

Key words and phrases: Noisy annotations, Classification, Imaging data, Functional data analysis, Label integration

Information

Preprint No.SS-2025-0231
Manuscript IDSS-2025-0231
Complete AuthorsShuoyang Wang, Grace Y Yi, Guanqun Cao
Corresponding AuthorsGuanqun Cao
Emailscaoguanq@msu.edu

References

  1. Arpit, D., S. Jastrz˛ebski, N. Ballas, D. Krueger, E. Bengio, M. S. Kanwal, T. Maharaj,
  2. A. Fischer, A. Courville, Y. Bengio, and S. Lacoste-Julien (2017). A closer look at memorization in deep networks. In International Conference on Machine Learning, pp. 233–242. PMLR.
  3. Asman, A. J. and B. A. Landman (2011). Robust statistical label fusion through consensus level, labeler accuracy, and truth estimation (collate). IEEE Transactions on Medical Imaging 30(10), 1779–1794.
  4. Asman, A. J. and B. A. Landman (2012). Formulating spatially varying performance in the statistical fusion framework. IEEE Transactions on Medical Imaging 31(6), 1326–1336.
  5. Bauer, B. and M. Kohler (2019). On deep learning as a remedy for the curse of dimensionality in nonparametric regression. The Annals of Statistics 47, 2261–2285.
  6. Berrendero, J. R., A. Cuevas, and J. L. Torrecilla (2018). On the use of reproducing kernel hilbert spaces in functional classification. Journal of the American Statistical Association 113(523), 1210–1218.
  7. Bos, T. and J. Schmidt-Hieber (2022). Convergence rates of deep relu networks for multiclass classification. Electronic Journal of Statistics 16, 2724–2773.
  8. Cai, T. T. and L. Zhang (2019a). A convex optimization approach to high-dimensional sparse quadratic discriminant analysis. arXiv:1912.02872.
  9. Cai, T. T. and L. Zhang (2019b). High dimensional linear discriminant analysis: optimality, adaptive algorithm and missing data. Journal of the Royal Statistical Society (Series
  10. B) 81(4), 675–705.
  11. Carroll, R. J., D. Ruppert, L. A. Stefanski, and C. M. Crainiceanu (2006). Measurement Error in Nonlinear Models: A Modern Perspective. Chapman and Hall/CRC.
  12. Chen, Z., H. Wang, H. Sun, P. Chen, T. Han, X. Liu, and J. Yang (2021). Structured probabilistic end-to-end learning from crowds. In Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence, pp. 1512–1518.
  13. Dai, X., H.-G. Müller, and F. Yao (2017). Optimal Bayes classifiers for functional data and density ratios. Biometrika 104(3), 545–560.
  14. Delaigle, A. and P. Hall (2012). Achieving near-perfect classification for functional data. Journal of the Royal Statistical Society (Series B) 74, 267–286.
  15. Grigsby, J. E., K. Lindsey, and D. Rolnick (2023). Hidden symmetries of relu networks. In A. C. Krause, E. Brunskill, K. Cho, B. Engelhardt, S. Sabato, and J. Scarlett (Eds.), Proceedings of the 40th International Conference on Machine Learning, Volume 202 of Proceedings of Machine Learning Research, pp. 11734–11760. PMLR.
  16. Guan, M., V. Gulshan, A. Dai, and G. Hinton (2018). Who said what: Modeling individual labelers improves classification. In Proceedings of the AAAI conference on artificial intelligence, Volume 32.
  17. Guo, H., B. Wang, and Y. G. Yi (2023). Label correction of crowdsourced noisy annotations with an instance-dependent noise transition model. Advances in Neural Information Processing Systems 36, 347–386.
  18. Han, B., I. W. Tsang, L. Chen, J. T. Zhou, and P. Y. Celina (2019). Beyond majority voting: A coarse-to-fine label filtration for heavily noisy labels. IEEE Transactions on Neural Networks and Learning Systems 30(12), 3774–3787.
  19. Hastie, T. J. and R. J. Tibshirani (1990). Generalized Additive Models. Chapman & Hall/CRC.
  20. Hu, T., Z. Shang, and G. Cheng (2020). Sharp rate of convergence for deep neural network classifiers under the teacher-student setting. arXiv:2001.06892.
  21. Huang, C. and H. Zhu (2022). Functional hybrid factor regression model for handling heterogeneity in imaging studies. Biometrika 109(4), 1133–1148.
  22. Ibrahim, S. and X. Fu (2021). Crowdsourcing via annotator co-occurrence imputation and provable symmetric nonnegative matrix factorization. In International Conference on Machine Learning, pp. 4544–4554. PMLR.
  23. Ibrahim, S., T. Nguyen, and X. Fu (2023). Deep learning from crowdsourced labels: Coupled cross-entropy minimization, identifiability, and regularization. In international Conference on Representation Learning.
  24. Khetan, A., Z. C. Lipton, and A. Anandkumar (2017). Learning from noisy singly-labeled data. arXiv preprint arXiv:1712.04577.
  25. Kim, Y., I. Ohn, and D. Kim (2021). Fast convergence rates of deep neural networks for classification. Neural Networks 138, 179–197.
  26. Li, H. and B. Yu (2014). Error rate bounds and iterative weighted majority voting for crowdsourcing. arXiv preprint arXiv:1411.4086.
  27. Lin, Y. (2000). Tensor product space anova models. The Annals of Statistics 28, 734 – 755.
  28. Liu, Q., J. Peng, and A. T. Ihler (2012). Variational inference for crowdsourcing. Advances in Neural Information Processing Systems 25.
  29. Liu, R., Z. Shang, and G. Cheng (2021). On deep instrumental variables estimate. arXiv:2004.14954.
  30. Mai, Q., Y. Yang, and H. Zou (2019). Multiclass sparse discriminant analysis. Statistica Sinica 29, 97–111.
  31. Mammen, E. and A. B. Tsybakov (1999). Smooth discrimination analysis. The Annals of Statistics 27, 1808–1829.
  32. Nishie, A., D. Kakihara, T. Nojo, K. Nakamura, S. Kuribayashi, M. Kadoya, K. Ohtomo,
  33. K. Sugimura, and H. Honda (2015). Current radiologist workload and the shortages in japan: how many full-time radiologists are required? Japanese Journal of Radiology 33, 266–272.
  34. Park, J., J. Ahn, and Y. Jeon (2021). Sparse functional linear discriminant analysis. Biometrika 109(1), 209–226.
  35. Raykar, V. C., S. Yu, L. H. Zhao, A. Jerebko, C. Florin, G. H. Valadez, L. Bogoni, and L. Moy
  36. (2009). Supervised learning from multiple experts: whom to trust when everyone lies a bit. In Proceedings of the 26th Annual International Conference on Machine Learning, pp. 889–896.
  37. Reed, S., H. Lee, D. Anguelov, C. Szegedy, D. Erhan, and A. Rabinovich (2014). Training deep neural networks on noisy labels with bootstrapping. arXiv preprint arXiv:1412.6596.
  38. Schmidt-Hieber, J. (2020). Nonparametric regression using deep neural networks with relu activation function. The Annals of Statistics 48(4), 1875–1897.
  39. Shieh, D. and R. T. Ogden (2023). Permutation-based inference for function-on-scalar regression with an application in PET brain imaging. Journal of Nonparametric Statistics 35(4), 820–838.
  40. Srivastava, N., G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov (2014). Dropout: a simple way to prevent neural networks from overfitting. Journal of Machine Learning Research 15, 1929–1958.
  41. Sukhbaatar, S., J. Bruna, M. Paluri, L. Bourdev, and R. Fergus (2014). Training convolutional networks with noisy labels. arXiv preprint arXiv:1406.2080.
  42. Sun, Y. and G. Y. Yi (2026a). Debiased estimation and variable selection under function-onscalar linear regression models with ultrahigh-dimensional covariates subject to measurement error. Bernoulli, To appear.
  43. Sun, Y. and G. Y. Yi (2026b). Estimation and variable selection under the function-on-scalar linear model with covrirate measurement error. Statistica Sinica 36(2).
  44. Tanno, R., A. Saeedi, S. Sankaranarayanan, C. D. Alexander, and N. Silberman (2019). Learning from noisy labels by regularized estimation of annotator confusion. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 11244–11253.
  45. Tsybakov, A. B. (2004). Optimal aggregation of classifiers in statistical learning. The Annals of Statistics 32, 135–166.
  46. Wang, S. and G. Cao (2024). Multiclass classification for multidimensional functional data through deep neural networks. Electronic Journal of Statistics 18, 1248–1292.
  47. Wang, S., G. Cao, and Z. Shang (2021). Estimation of the mean function of functional data via deep neural networks. Stat e393.
  48. Wang, S., G. Cao, and Z. Shang (2023). Deep neural network classifier for multi-dimensional functional data. Scandinavian Journal of Statistics 50, 1667–1686.
  49. Wang, S., Z. Shang, G. Cao, and S. J. Liu (2024). Optimal classification for functional data. Statistica Sinica 34, 1545–1564.
  50. Wang, Y., G. Wang, L. Wang, and R. T. Ogden (2020). Simultaneous confidence corridors for mean functions in functional data analysis of imaging data. Biometrics 76(2), 427–437.
  51. Warfield, S. K., K. H. Zou, and W. M. Wells (2004). Simultaneous truth and performance level estimation (staple): an algorithm for the validation of image segmentation. IEEE Transactions on Medical Imaging 23(7), 903–921.
  52. Wei, H., R. Xie, L. Feng, B. Han, and B. An (2022). Deep learning from multiple noisy annotators as a union. IEEE Transactions on Neural Networks and Learning Systems 34(12), 10552–10562.
  53. Whitehill, J., T.-f. Wu, J. Bergsma, J. Movellan, and P. Ruvolo (2009). Whose vote should count more: Optimal integration of labels from labelers of unknown expertise. Advances in Neural Information Processing Systems 22.
  54. Witten, D. M. and R. Tibshirani (2011). Penalized classification using Fisher’s linear dis38 criminant. Journal of the Royal Statistical Society. Series B. Statistical Methodology 73(5), 753–772.
  55. Yi, G. Y. (2017). Statistical Analysis with Measurement Error or Misclassification: Strategy, Method and Application. Springer, New York.
  56. Yi, G. Y., A. Delaigle, and P. Gustafson (2021). Handbook of measurement error models. Chapman & Hall/CRC, Boca Raton, FL.
  57. Zhang, L., R. Tanno, M.-C. Xu, C. Jin, J. Jacob, O. Ciccarelli, F. Barkhof, and C. D. Alexander
  58. (2020). Disentangling human error from the ground truth in segmentation of medical images. Advances in Neural Information Processing Systems 1321, 15750–15762.
  59. Zhang, Y., X. Chen, D. Zhou, and M. I. Jordan (2016). Spectral methods meet em: A provably optimal algorithm for crowdsourcing. Journal of Machine Learning Research 17(102), 1–44.
  60. Zhu, H., J. Fan, and L. Kong (2014). Spatially varying coefficient model for neuroimaging data with jump discontinuities. Journal of the American Statistical Association 109(507), 1084–1098.

Acknowledgments

We thank the Editor, the Associate Editor, and the two anonymous reviewers for their constructive comments on the initial submission.

Grace Y. Yi is Canada Research Chair in Data Science (Tier 1). Her research was

supported by the Natural Sciences and Engineering Research Council of Canada (NSERC)

and the Canada Research Chair Program. Guanqun Cao’s research is partially supported by the

National Science Foundation under Grants DMS-2413301, CNS-2319342 and CNS-2319343.

ADNI data used our analysis of this article were obtained from the Alzheimers Disease

Neuroimaging Initiative (ADNI) database (adni.loni.usc.edu). As such, the investigators within the ADNI contributed to the design and implementation of ADNI and/or

provided data but did not participate in analysis or writing of this report. A complete listing of

ADNI investigators can be found at: http://adni.loni.usc.edu/wp-content/

uploads/how_to_apply/ADNI_Acknowledgement_List.pdf.

Supplementary Materials

.

5.2. Simulation studies

In this section, we present numerical results to demonstrate the superior performance of the

proposed afDNN method. We consider two types of annotators as considered by Tanno et al.

Supplementary materials are available for download.