An apparent inconsistency arising from tests of significance based on fiducial distributions of unknown parameters

Abstract

It has been observed that the Behrens and Fisher test of the difference of the means of two samples gives a smaller percentage of significant results than might be expected on the analogy of the ordinary t test with a pooled estimate of variance. The cause of this apparent anomaly is explained, and it is shown that the criticisms of the test to which the anomaly has given rise have their origin in (a) neglect of the relevant information provided by the estimated values of the variances, and (b) an insufficient appreciation of the fiducial basis of all tests of significance (including the ordinary t test) on small samples.It is pointed out that Sukhatme's table (constructed for the Behrens and Fisher test) also provides a test for the weighted mean of the means of two sets of observations, concerning whose relative accuracy no prior knowledge is available.