Abstract

Conjugate distributions provide an entry point to Bayesian analysis.

By defining summation, subtraction, and multiplication operators for conjugate

distributions, we study Bayesian statistics by arithmetic operations. A striking

feature is that the non-informative prior fulfills the central role of zero in mathematics. The summation operator connects Bayesian and frequentist estimators

by a simple equation, which also provides an efficient method for evaluating the

marginal likelihood. The subtraction operator facilitates cross-validation, rollingwindow estimation, and regression under multicollinearity. The multiplication

operator simplifies the weighted regression with a discount factor. Arithmetic

operations conceptualize pseudo data in the conjugate prior, sufficient statistics

that determine the likelihood, and the posterior that balances the prior and data.

Key words and phrases: Conjugacy, Exponential family, Linear regression, Statis- tics education 1

Information

Preprint No.SS-2024-0052
Manuscript IDSS-2024-0052
Complete AuthorsHang Qian
Corresponding AuthorsHang Qian
Emailsmatlabist@gmail.com

References

  1. Bernardo, J. and A. Smith (2000). Bayesian Theory. Chichester: John Wiley and Sons.
  2. Bolstad, W. and J. Curran (2017). Introduction to Bayesian Statistics (Third Edition). Hoboken: Wiley.
  3. Christensen, R., W. Johnson, A. Branscum, and T. Hanson (2011). Bayesian Ideas and Data Analysis: An Introduction for Scientists and Statisticians. Boca Raton: CRC Press.
  4. Diaconis, P. and D. Ylvisaker (1979). Conjugate priors for exponential families. Annals of Statistics 7, 269–281.
  5. Fama, E. F. and K. R. French (2015). A five-factor asset pricing model. Journal of Financial Economics 116, 1–22.
  6. Gelman, A. (2008). Objections to Bayesian statistics. Bayesian Analysis 3, 445–449.
  7. Gelman, A., J. Carlin, H. Stern, D. Dunson, A. Vehtari, and D. Rubin
  8. (2014). Bayesian Data Analysis (Third Edition). Boca Raton: CRC Press.
  9. Koop, G. (2003). Bayesian Econometrics. Hoboken: Wiley.
  10. Park, T. and G. Casella (2008). The Bayesian Lasso. Journal of the American Statistical Association 103(482), 681–686.
  11. Qian, H. (2018). Big data Bayesian linear regression and variable selection by normal-inverse-gamma summation. Bayesian Analysis 13(4), 1011– 1035.
  12. Tibshirani, R. (1996). Regression shrinkage and selection via the Lasso. Journal of the Royal Statistical Society B 58, 267–288.

Acknowledgments

I would like to thank the editors for suggestions regarding arithmetic operators for the exponential family. The feedback greatly improved the quality

of this paper.