Abstract: Conjugate distributions provide an entry point to Bayesian analysis. By defining summation, subtraction, and multiplication operators for conjugate distributions, we study Bayesian statistics by arithmetic operations. A striking feature is that the non-informative prior fulfills the central role of zero in mathematics. The summation operator connects Bayesian and frequentist estimators by a simple equation, which also provides an efficient method for evaluating the marginal likelihood. The subtraction operator facilitates cross-validation, rolling-window estimation, and regression under multicollinearity. The multiplication operator simplifies the weighted regression with a discount factor. Arithmetic operations conceptualize pseudo data in the conjugate prior, sufficient statistics that determine the likelihood, and the posterior that balances the prior and data.

Key words and phrases: Conjugacy, exponential family, linear regression, statistics education.