Abstract

We develop a novel approach to address the common but challenging

problem of conformal inference for missing data in machine learning, focusing

on Missing at Random (MAR) data. We propose a new procedure, Conformal

Prediction for Missing Data under Multiple Robust Learning (CM–MRL), which

combines split conformal calibration with a multiple robust empirical-likelihood

(EL) reweighting scheme.

The method proceeds via a double calibration by

reweighting the complete-case scores by EL so that their distribution matches the

full calibration distribution implied by MAR, even when some working models

are misspecified.

We demonstrate the asymptotic behavior of our estimators

through empirical process theory, provide reliable coverage for our prediction

intervals, both marginally and conditionally, and further show an interval-length

dominance result.

We demonstrate the effectiveness of the proposed method

through several numerical experiments in the presence of missing data.

Key words and phrases: Conformal inference, Missing data, Multiple robust model, Uncertainty quantification, Missing at random

Information

Preprint No.SS-2025-0389
Manuscript IDSS-2025-0389
Complete AuthorsWenlu Tang, Hongni Wang, Xingcai Zhou, Bei Jiang, Linglong Kong
Corresponding AuthorsLinglong Kong
Emailslkong@ualberta.ca

References

  1. Cand`es, E., L. Lei, and Z. Ren (2023). Conformalized survival analysis. Journal of the Royal Statistical Society Series B: Statistical Methodology 85(1), 24–45.
  2. Cao, W., A. A. Tsiatis, and M. Davidian (2009). Improving efficiency and robustness of the doubly robust estimator for a population mean with incomplete data. Biometrika 96(3), 723–734.
  3. Chernozhukov, V., D. Chetverikov, M. Demirer, E. Duflo, C. Hansen, W. Newey, and J. Robins
  4. (2018, 01). Double/debiased machine learning for treatment and structural parameters. The Econometrics Journal 21(1), C1–C68.
  5. Feldman, S., S. Bates, and Y. Romano (2021). Improving conditional coverage via orthogonal quantile regression. Advances in neural information processing systems 34, 2060–2071.
  6. Foygel Barber, R., E. J. Candes, A. Ramdas, and R. J. Tibshirani (2021). The limits of distribution-free conditional predictive inference. Information and Inference: A Journal of the IMA 10(2), 455–482.
  7. Gibbs, I. and E. Candes (2021). Adaptive conformal inference under distribution shift. Advances in Neural Information Processing Systems 34, 1660–1672.
  8. Hammer, S. M., D. A. Katzenstein, M. D. Hughes, H. Gundacker, R. T. Schooley, R. H.
  9. Haubrich, W. K. Henry, M. M. Lederman, J. P. Phair, M. Niu, et al. (1996). A trial comparing nucleoside monotherapy with combination therapy in hiv-infected adults with cd4 cell counts from 200 to 500 per cubic millimeter. New England Journal of Medicine 335(15), 1081–1090.
  10. Han, P. (2014a). A further study of the multiply robust estimator in missing data analysis.
  11. Journal of Statistical Planning and Inference 148, 101–110.
  12. Han, P. (2014b). Multiply robust estimation in regression analysis with missing data. Journal of the American Statistical Association 109(507), 1159–1173.
  13. Han, P. (2016). Intrinsic efficiency and multiple robustness in longitudinal studies with drop-out. Biometrika 103(3), 683–700.
  14. Han, P., L. Kong, J. Zhao, and X. Zhou (2019). A general framework for quantile estimation with incomplete data. Journal of the Royal Statistical Society Series B: Statistical Methodology 81(2), 305–333.
  15. Han, P. and L. Wang (2013). Estimation with missing data: beyond double robustness. Biometrika 100(2), 417–430.
  16. Kang, J. D. and J. L. Schafer (2007). Demystifying double robustness: A comparison of alternative strategies for estimating a population mean from incomplete data.
  17. Lei, J., M. G’Sell, A. Rinaldo, R. J. Tibshirani, and L. Wasserman (2018). Distribution-free predictive inference for regression. Journal of the American Statistical Association 113(523), 1094–1111.
  18. Lei, J., J. Robins, and L. Wasserman (2013). Distribution-free prediction sets. Journal of the American Statistical Association 108(501), 278–287.
  19. Lei, J. and L. Wasserman (2014). Distribution-free prediction bands for non-parametric regression. Journal of the Royal Statistical Society: Series B: Statistical Methodology, 71–96.
  20. Lei, L. and E. J. Cand`es (2021). Conformal inference of counterfactuals and individual treatment effects. Journal of the Royal Statistical Society Series B: Statistical Methodology 83(5), 911–938.
  21. Li, W., Y. Gu, and L. Liu (2020). Demystifying a class of multiply robust estimators. Biometrika 107(4), 919–933.
  22. Little, R. J. and D. B. Rubin (2002). Maximum likelihood for general patterns of missing data: Introduction and theory with ignorable nonresponse. Statistical analysis with missing data, 164–189.
  23. Little, R. J. and D. B. Rubin (2019). Statistical analysis with missing data, Volume 793. John Wiley & Sons.
  24. Robins, J. M. and A. Rotnitzky (1995). Semiparametric efficiency in multivariate regression models with missing data. Journal of the American Statistical Association 90(429), 122– 129.
  25. Romano, Y., E. Patterson, and E. Candes (2019). Conformalized quantile regression. Advances in Neural Information Processing Systems 32, 3543–3553.
  26. Rosenbaum, P. R. and D. B. Rubin (1983). The central role of the propensity score in observational studies for causal effects. Biometrika 70(1), 41–55.
  27. Rubin, D. B. (1996). Multiple imputation after 18+ years. Journal of the American statistical Association 91(434), 473–489.
  28. Rubin, D. B. (2018). Multiple imputation. In Flexible Imputation of Missing Data, Second Edition, pp. 29–62. Chapman and Hall/CRC.
  29. Sesia, M. and Y. Romano (2021). Conformal prediction using conditional histograms. Advances in Neural Information Processing Systems 34, 6304–6315.
  30. Tan, Z. (2010). Bounded, efficient and doubly robust estimation with inverse weighting. Biometrika 97(3), 661–682.
  31. Tibshirani, R. J., R. Foygel Barber, E. Candes, and A. Ramdas (2019). Conformal prediction under covariate shift. Advances in neural information processing systems 32.
  32. Tu, R., C. Zhang, P. Ackermann, K. Mohan, H. Kjellstr¨om, and K. Zhang (2019). Causal discovery in the presence of missing data. In The 22nd International Conference on Artificial Intelligence and Statistics, pp. 1762–1770. PMLR.
  33. Vovk, V. (2012). Conditional validity of inductive conformal predictors. In Asian conference on machine learning, pp. 475–490. PMLR.
  34. Vovk, V., A. Gammerman, and G. Shafer (2005). Algorithmic learning in a random world. Springer Science & Business Media.
  35. Vovk, V., I. Petej, P. Toccaceli, A. Gammerman, E. Ahlberg, and L. Carlsson (2020). Conformal calibrators. In conformal and probabilistic prediction and applications, pp. 84–99. PMLR.
  36. Zaffran, M., A. Dieuleveut, J. Josse, and Y. Romano (2023). Conformal prediction with missing values. arXiv preprint arXiv:2306.02732.
  37. Zaffran, M., O. F´eron, Y. Goude, J. Josse, and A. Dieuleveut (2022). Adaptive conformal predictions for time series. In International Conference on Machine Learning, pp. 25834– 25866. PMLR. Wenlu Tang: University of Alberta

Supplementary Materials

In the Supplementary Materials, we present extensions of the proposed

method, including the algorithm pseudocode (S1.1), detailed experiment

settings (S1.2, S1.3), extensions under model misspecification (S1.4), the

quantile conformity score version (S1.5), and the double machine learning

extension (S1.6). We also include technical proofs of the main results (S2).

Supplementary materials are available for download.