Abstract

Assessing causal effects of interventions on ordinal outcomes is essential in economics, biological experiments, and behavioral studies. In the potential outcomes framework, average treatment effects are defined by comparing outcomes under treatment and control conditions, but ordinal outcomes are often difficult to interpret. To address this, we consider the relative treatment effect, defined as the difference between the probability that the treatment is beneficial and the probability that it is harmful. However, it depends on the association between potential outcomes, which is not identifiable from the observed data. We derive sharp bounds on the relative treatment effect within a stratified randomization framework, given the marginal distributions of the potential outcomes. Moreover, we allow for the special case of a single stratum, and generalize the approach to handle scenarios with binary and multi-category potential outcomes. We enhance estimation precision through covariate adjustment and further narrow the bounds, and also consider cases with independent potential outcomes. We construct nonparametric bootstrap confidence intervals for sharp bounds and evaluate the proposed estimator via numerical simulations and an empirical analysis.

Key words and phrases: Causal inference, covariate adjustment, ordinal outcomes, partial identification, stratification, sharp bound

Information

Preprint No.SS-2025-0495
Manuscript IDSS-2025-0495
Complete AuthorsGuangyuran Liao, Yuehan Yang
Corresponding AuthorsYuehan Yang
Emailsyyh@cufe.edu.cn

References

  1. Agresti, A. (2010). Analysis of ordinal categorical data, Volume 656. John Wiley & Sons.
  2. Agresti, A. and M. Kateri (2017). Ordinal probability effect measures for group comparisons in multinomial cumulative link models. Biometrics 73(1), 214–219.
  3. Balke, A. and J. Pearl (1997). Bounds on treatment effects from studies with imperfect compliance. Journal of the American statistical Association 92(439), 1171–1176.
  4. Bloniarz, A., H. Liu, C.-H. Zhang, J. S. Sekhon, and B. Yu (2016). Lasso adjustments of treatment effect estimates in randomized experiments. Proceedings of the National Academy of Sciences 113(27), 7383–7390.
  5. Cheng, J. (2009). Estimation and inference for the causal effect of receiving treatment on a multinomial outcome. Biometrics 65(1), 96–103.
  6. Chiba, Y. (2018). Bayesian inference of causal effects for an ordinal outcome in randomized trials. Journal of Causal Inference 6(2), 20170019.
  7. Ciolino, J. D., H. L. Palac, A. Yang, M. Vaca, and H. M. Belli (2019). Ideal vs. real: a systematic review on handling covariates in randomized controlled trials. BMC medical research methodology 19(1), 136.
  8. D´ıaz, I., E. Colantuoni, and M. Rosenblum (2016). Enhanced precision in the analysis of randomized trials with ordinal outcomes. Biometrics 72(2), 422–431.
  9. Ding, P. and T. Dasgupta (2016). A potential tale of two-by-two tables from completely randomized experiments. Journal of the American Statistical Association 111(513), 157–168.
  10. Ding, P., A. Feller, and L. Miratrix (2019). Decomposing treatment effect variation. Journal of the American Statistical Association 114(525), 304–317.
  11. D’Amico, G., J. G. Abraldes, P. Rebora, M. G. Valsecchi, and G. Garcia-Tsao (2020). Ordinal outcomes are superior to binary outcomes for designing and evaluating clinical trials in compensated cirrhosis. Hepatology 72(3), 1029–1042.
  12. Fisher, R. A. (1992). The arrangement of field experiments. In Breakthroughs in statistics: Methodology and distribution, pp. 82–91. Springer.
  13. Frandsen, B. R. and L. J. Lefgren (2021). Partial identification of the distribution of treatment effects with an application to the knowledge is power program (kipp). Quantitative Economics 12(1), 143–171.
  14. Fr´echet, M. (1935). G´en´eralisation du th´eoreme des probabilit´es totales. Fundamenta mathematicae 25(1), 379–387.
  15. Freedman, D. A. (2008). On regression adjustments to experimental data. Advances in Applied Mathematics 40(2), 180–193.
  16. Hoff, P. D. (2009). A first course in Bayesian statistical methods, Volume 580. Springer.
  17. Horowitz, J. L. and C. F. Manski (2000). Nonparametric analysis of randomized experiments with missing covariate and outcome data. Journal of the American Statistical Association 95(449), 77–84.
  18. Huang, E. J., E. X. Fang, D. F. Hanley, and M. Rosenblum (2017). Inequality in treatment benefits: Can we determine if a new treatment benefits the many or the few? Biostatistics 18(2), 308–324.
  19. Imai, K. (2008). Variance identification and efficiency analysis in randomized experiments under the matched-pair design. Statistics in medicine 27(24), 4857–4873.
  20. Imbens, G. W. and D. B. Rubin (2015). Causal inference in statistics, social, and biomedical sciences. Cambridge university press.
  21. Jiang, Z. and P. Ding (2018). Using missing types to improve partial identification with application to a study of hiv prevalence in malawi. The Annals of Applied Statistics 12(3), 1831–1852.
  22. Lee, D. S. (2009). Training, wages, and sample selection: Estimating sharp bounds on treatment effects. Review of Economic Studies 76(3), 1071–1102.
  23. Lei, L. and P. Ding (2021). Regression adjustment in completely randomized experiments with a diverging number of covariates. Biometrika 108(4), 815–828.
  24. Li, B., H. F. Lingsma, E. W. Steyerberg, and E. Lesaffre (2011). Logistic random effects regression models: a comparison of statistical packages for binary and ordinal outcomes. BMC medical research methodology 11, 1–11.
  25. Lin, W. (2013). Agnostic notes on regression adjustments to experimental data: Reexamining freedman’s critique. The Annals of Applied Statistics, 295–318.
  26. Lin, Y., M. Zhu, and Z. Su (2015). The pursuit of balance: an overview of covariate-adaptive randomization techniques in clinical trials. Contemporary clinical trials 45, 21–25.
  27. Liu, H. and Y. Yang (2020). Regression-adjusted average treatment effect estimates in stratified randomized experiments. Biometrika 107(4), 935–948.
  28. Lu, J., P. Ding, and T. Dasgupta (2018). Treatment effects on ordinal outcomes: Causal estimands and sharp bounds. Journal of Educational and Behavioral Statistics 43(5), 540–567.
  29. Lu, J., Y. Zhang, and P. Ding (2020). Sharp bounds on the relative treatment effect for ordinal outcomes. Biometrics 76(2), 664–669.
  30. Ma, W., Y. Qin, Y. Li, and F. Hu (2020). Statistical inference for covariate-adaptive randomization procedures. Journal of the American Statistical Association 115(531), 1488–1497.
  31. Manski, C. F. (2003). Partial identification of probability distributions. Springer Science & Business Media.
  32. Miao, W., W. Hu, E. L. Ogburn, and X.-H. Zhou (2023). Identifying effects of multiple treatments in the presence of unmeasured confounding. Journal of the American Statistical Association 118(543), 1953–1967.
  33. Miratrix, L. W., J. S. Sekhon, and B. Yu (2013). Adjusting treatment effect estimates by post-stratification in randomized experiments. Journal of the Royal Statistical Society Series B: Statistical Methodology 75(2), 369– 396.
  34. Morgan, K. L. and D. B. Rubin (2015). Rerandomization to balance tiers of covariates. Journal of the American Statistical Association 110(512), 1412–1421.
  35. Praet, M. and A. Desoete (2014). Enhancing young children’s arithmetic skills through non-intensive, computerised kindergarten interventions: A randomised controlled study. Teaching and Teacher Education 39, 56–65.
  36. Richardson, A., M. G. Hudgens, P. B. Gilbert, and J. P. Fine (2014). Nonparametric bounds and sensitivity analysis of treatment effects. Statistical science: a review journal of the Institute of Mathematical Statistics 29(4), 596.
  37. Rosenberger, W. F. and O. Sverdlov (2008). Handling covariates in the design of clinical trials. Statistical Science 23(3), 404–419.
  38. Rubin, D. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology 66(5), 688.
  39. Rubin, D. B. (1978). Bayesian inference for causal effects: The role of randomization. The Annals of statistics, 34–58.
  40. Rubin, D. B. (1980). Randomization analysis of experimental data: The fisher randomization test comment. Journal of the American Statistical Association 75(371), 591–593.
  41. Rubin, D. B. (2008). For objective causal inference, design trumps analysis. The Annals of Applied Statistics, 808–840.
  42. R¨uschendorf, L. (1981). Sharpness of fr´echet-bounds. Zeitschrift f¨ur Wahrscheinlichkeitstheorie und verwandte Gebiete 57(2), 293–302.
  43. Scott, S. C., M. S. Goldberg, and N. E. Mayo (1997). Statistical assessment of ordinal outcomes in comparative studies. Journal of clinical epidemiology 50(1), 45–55.
  44. Splawa-Neyman, J., D. M. Dabrowska, and T. P. Speed (1990). On the application of probability theory to agricultural experiments. essay on principles. section 9. Statistical Science, 465–472.
  45. Strassen, V. (1965). The existence of probability measures with given marginals. The Annals of Mathematical Statistics 36(2), 423–439.
  46. Tu, F., W. Ma, and H. Liu (2024). A unified framework for covariate adjustment under stratified randomisation. Stat 13(4), e70016.
  47. Volfovsky, A., E. M. Airoldi, and D. B. Rubin (2015). Causal inference for ordinal outcomes. arXiv preprint arXiv:1501.01234.
  48. Wu, P., P. Ding, Z. Geng, and Y. Liu (2024). Quantifying individual risk for binary outcome. arXiv preprint arXiv:2402.10537.
  49. Yue, L., G. Li, H. Lian, and X. Wan (2019). Regression adjustment for treatment effect with multicollinearity in high dimensions. Computational Statistics & Data Analysis 134, 17–35.
  50. Zhang, C., Z. Geng, W. Li, and P. Ding (2024). Identifying and bounding the probability of necessity for causes of effects with ordinal outcomes. arXiv preprint arXiv:2411.01234.
  51. Zhao, A. and P. Ding (2024). To adjust or not to adjust? estimating the average treatment effect in randomized experiments with missing covariates. Journal of the American Statistical Association 119(545), 450–460. Guangyuran Liao

Supplementary Materials

The online Supplementary Material contains proofs for Proposition 1-4, Theorems 1-3, and Corollary 1.

Supplementary materials are available for download.