An Evaluator's Guide To Detecting Attrition Problems

Abstract

This article reviews simple methods for detecting problematic attrition (nonresponse) in longitudinal evaluations. It begins by discussing nonresponse more generally and considers three situations that the evaluator is likely to encounter: (a) simple cross-sectional comparisons of treatment and control groups, (b) regression-based analyses of cross-sectional data on treatment and control groups, and (c) regression-based analyses of data from a longitudinal evaluation. The last is the focus of this article; the first 2 are discussed to provide the reader with the necessary background The article then discusses means of detecting attrition problems in longitudinal evaluations Model-based solutions are computationally demanding, but the article discusses simple tests for detecting problematic attrition. These methods are then illustrated using data from the Fort Bragg Evaluation, a longitudinal evaluation of a major demonstration in the field of children's mental health services. The analyses presented here do not suggest that attrition distorted the findings of that evaluation.