Application of a Measure of Information to the Design and Comparison of Regression Experiments

Abstract

A normal regression experiment can be represented by egin{equation*} ag{1.1} Y_i = sum_{j=1}^k X_{ij} heta_j + eta_iqquad (i = 1, cdots, n)end{equation*} where ${eta_i/i = 1, cdots, n}$ is a set of normally distributed random variables with zero means and non-singular dispersion matrix $C, heta = ( heta_1, cdots, heta_k)$ is the parameter-vector of interest and $X = (X_{ij})$ is a known $n imes k$ matrix which will be called the allocation matrix. The rows of $X$ will be called the allocation vectors. We denote the experiment by $varepsilon(X, C)$. We assume that $C$ is known; generally it will be a function of $X, C(X)$. The particular realisation of $Y$ will be denoted $y$. The matrix $F = X'C^{-1}X$ is the Fisher-information-matrix of $varepsilon(X, C)$. When $F$ is non-singular, one answer to the question "What information does $y$ give about $ heta$?" is to quote $F^{-1}$, the dispersion matrix of the maximum-likelihood-estimates of $ heta$. A strong argument in favour of this is that $F^{-1}$ is independent of both $ heta$ and $y$. The fact that it is independent of $ heta$ means that the answer is not "local"; the fact that it is independent of $y$ leads to simplicity. This approach is taken by Box and Hunter [1] in their work on rotatable designs. However, we must accept the fact that many experimenters wish to have a one-dimensional answer to the question i.e. we must associate with $varepsilon(X, C)$ a single number which we call the "information". For instance Elfving [5] has developed the use of trace $F^{-1}$. In this paper we adopt the measure of information introduced by Lindley [7]. In Section 2 we generalise Lindley's treatment of the regression situation to include the singular case, explain the uses of the measure and compare it with that of Elfving. Section 3 deals with the analogue of Elfving's main theorem. Theorems 4.1 and 4.2 of Section 4 provide links with the traditional variance approach. In Section 5 we derive the asymptotic form of the measure as the $n$ of (1.1) increases and show that this form can be derived also from Neyman-Pearsonian theory. In Section 6 the influence of nuisance parameters is discussed and an analogue of a theorem of Chernoff [2] is established.