A Note on Exact Tests and Confidence Intervals for Two-by-Two Contingency Tables in Randomized Trials

There are two major exact tests used for hypothesis testing of two-by-two contingency tables: Fisher’s exact test and Barnard’s exact test. Recently, Chiba (Journal of Biometrics and Biostatistics 2015; 2: 244) developed new exact tests: a conditional exact test, which requires that a marginal total is fixed, and an unconditional exact test, which does not require that a marginal total is fixed and depends rather on the ratio of random assignment. Fisher’s exact test can be regarded as a special case of the conditional exact test. For Barnard’s and Chiba’s exact tests, the confidence intervals linking to them can be constructed in a straightforward manner. In this article, we review these three exact tests, noting the differences in the null hypotheses that they test. Furthermore, using a numerical example, we demonstrate that the confidence interval linking to Barnard’s exact test is not in fact an exact confidence interval for the causal effect.