Statistica Sinica 11(2001), 807-826

AN OPTIMALITY THEORY FOR MID $p$-VALUES

IN $2 \times 2$ CONTINGENCY TABLES

J. T. Gene Hwang and Ming-Chung Yang

Cornell University and National Central University

Abstract: The contingency table arises in nearly every application of statistics. However, even the basic problem of testing independence is not totally resolved. More than thirty-five years ago, Lancaster (1961) proposed using the mid $p$-value for testing independence in a contingency table. The mid $p$-value is defined as half the conditional probability of the observed statistic plus the conditional probability of more extreme values, given the marginal totals. Recently there seems to be recognition that the mid $p$-value is quite an attractive procedure. It tends to be less conservative than the $p$-value derived from Fisher's exact test. However, the procedure is considered to be somewhat ad-hoc.

In this paper we provide theory to justify mid $p$-values. We apply the Neyman-Pearson fundamental lemma and the estimated truth approach, to derive optimal procedures, named expected $p$-values. The estimated truth approach views $p$-values as estimators of the truth function which is one or zero depending on whether the null hypothesis holds or not. A decision theory approach is taken to compare the $p$-values using risk functions. In the one-sided case, the expected $p$-value is exactly the mid $p$-value. For the two-sided case, the expected $p$-value is a new procedure that can be constructed numerically. In a contingency table of two independent binomial samplings with balanced sample sizes, the expected $p$-value reduces to a two-sided mid $p$-value. Further, numerical evidence shows that the expected $p$-values lead to tests which have type one error very close to the nomial level. Our theory provides strong support for mid $p$-values.

Key words and phrases: Estimated truth approach, Fisher's exact test, expected P-value.