Value-Added Analysis: A Dynamic Approach to the Estimation of Treatment Effects

Abstract

Randomized experiments are rarely feasible in large-scale educational and social evaluations. Most evaluations are observational studies in which the investigators have very limited control over the assignment of individuals to treatments. Since the effect of an intervention becomes confounded with those of other influences, clear causal inferences are very difficult to obtain. A number of statistical “adjustment” strategies have been suggested in an attempt to remove the bias attributable to these confounding factors. These techniques are based on a static model in which the outcome is a relatively simple function of various inputs, one of which is the treatment. In educational studies, one of the principal “inputs” is generally a measure of the outcome variable prior to the program. Statistical adjustments are based primarily on the observed relationship between these pretest scores and the posttest scores measured after the intervention. An alternative analysis approach is presented in this paper. It focuses explicitly on the fact that an educational treatment typically involves an intervention in a growth process. By modelling this process, it may be possible to estimate expected growth for various treatment groups under “control” conditions. Actual growth can be compared with projected growth to estimate thevalue-added by the program. A very simple model is developed here as a first step toward implementing this approach. The model is applied to a set of data taken from the Head Start Planned Variation Study. The results are compared with a parallel analysis using the analysis of covariance. Discrepancies between the results of the two analyses are examined, and it is suggested that the value-added results are more accurate in this instance.