An Evaluation of Methods for Estimating the Area Under the Receiver Operating Characteristic (ROC) Curve

Abstract

The area under the receiver operating characteristic (ROC) curve serves as one means for evaluating the performance of diagnostic and predictive test systems. The most commonly used method for estimating the area under an ROC curve utilizes the maximum-likelihood-estimation technique, and a nonparametric method to calculate the area under an ROC curve was recently described. We compared the performance of these two methods. The results for the area under the ROC curve and the standard error of the estimate as calculated by each of the two methods exhibited high correlation. Generally, the nonparametric method yields lower area estimates than the maximum-likelihood-estimation technique. However, these differences generally were small, particularly with ROC curves derived from five or more cutoff points. Consistent results of hypothesis testing of the significance of differences between two ROC curves will be similar, regardless of which method is used, as long as one uses the same estimation technique on the two curves and as long as the two ROC curves being compared are of similar shape.