Issues of Variability and Bias Affecting Multisite Measurement of Quality of Care

Abstract

Using data from a randomized trial to improve the quality of ambulatory care, the authors quantify the various sources of variability and bias that affect measures of quality of care and suggest experimental designs and analyses that reduce both bias and variability. There is a growing desire among health care researchers and government agencies to profile and compare practitioner performance. Such efforts are complicated by extreme inherent variability in most measures of quality of care, as well as potential biases introduced by "experiments," where patients cannot act as the unit of randomization. When the authors measured practitioner performance for eight patient-care guidelines, they found little association of level of performance across guidelines. Thus, the authors considered performance for each guideline separately, also taking into account variability between patients, practitioners, and practice conditions. Randomization can reduce bias in large studies but should be supplemented by multivariate models. A preintervention and postintervention design can reduce variability, but much of the variability that remains is because of unmeasured patient/error variance. Incorporation of these concepts into future studies using quality measurements will help researchers design smaller and more sensitive trials to draw more accurate and precise conclusions.