Presentation of comparative audit data

Abstract

The Royal College of Surgeons of England organizes a voluntary comparative audit service for its general surgical fellows. To date, the information from the audit has been presented purely descriptively, which does not take account of statistical variability and the effects of differences in case mix between the participants. A new approach to the presentation of these data, based on statistical modelling of the effects of case mix, is presented. The approach is illustrated in an application based on data relating to 136203 admissions reported by 110 surgeons. The result of adjusting for case mix is a substantial change in the relative performance of the participating surgeons, which demonstrates the folly of attempting to compare surgical performance on the basis of a measure as crude as the unadjusted mortality rate. Further development of the statistical modelling is required, but this new statistical approach appears to enhance greatly the credibility of the audit data, by quantifying its strengths and limitations.