On the Logical Justification of Conditional Tests for Two-By-Two Contingency Tables

Abstract

After a half-century of debate, there still seems to be no consensus on when to treat the margins of a two-by-two contingency table as fixed when conducting significance tests. I review some examples which indicate that causal models should play a prominent role in the debate. Under certain models for causal effects, unconditional tests can lead to the absurdity of inferring that a treatment will affect some population members, even though one cannot infer that the treatment affected any sample members. In such cases, conditional tests may be justified by the need to maintain coherence between sample and population inferences; there is no need to invoke concepts such as ancillarity, conditionality, or marginal information. Nevertheless, this justification does not apply beyond the specified causal models. I suggest that characterization of the two-by-two table problem as “a comparison of independent binomial proportions” may be too incomplete to compellingly justify any single approach to testing.