Power comparison of two‐sided exact tests for association in 2 × 2 contingency tables using standard, mid p and randomized test versions

Exact Pearson's chi square, likelihood ratio (LR), and Fisher's tests are obtained from the conditional distribution of its test statistic, given the row and column sums of the contingency table. The power and obtained significance level of the standard, mid p, and randomized versions of these tests are compared for two-sided tests in 2 x 2 tables, using binomial and multinomial sampling. The mid p type I error probabilities seldom exceed the nominal significance level. The mid p and randomized test versions have approximately the same power, and higher power than the standard test version. The power of the Pearson's chi square, LR and Fisher's test differ, and they differ in approximately the same way for standard, mid p and randomized test versions for any given set of parameters. There is no general ranking between the three tests. In many cases, Pearson's chi square and Fisher's tests have almost equal power, and higher power than LR. In a few cases, perhaps characterized by poorly balanced designs, LR performs best. Fisher's test seems to be slightly more robust even if the design is poor.