Evaluating the Incremental Value of New Biomarkers With Integrated Discrimination Improvement
Open Access
- 14 June 2011
- journal article
- research article
- Published by Oxford University Press (OUP) in American Journal of Epidemiology
- Vol. 174 (3), 364-374
- https://doi.org/10.1093/aje/kwr086
Abstract
The integrated discrimination improvement (IDI) index is a popular tool for evaluating the capacity of a marker to predict a binary outcome of interest. Recent reports have proposed that the IDI is more sensitive than other metrics for identifying useful predictive markers. In this article, the authors use simulated data sets and theoretical analysis to investigate the statistical properties of the IDI. The authors consider the common situation in which a risk model is fitted to a data set with and without the new, candidate predictor(s). Results demonstrate that the published method of estimating the standard error of an IDI estimate tends to underestimate the error. The z test proposed in the literature for IDI-based testing of a new biomarker is not valid, because the null distribution of the test statistic is not standard normal, even in large samples. If a test for the incremental value of a marker is desired, the authors recommend the test based on the model. For investigators who find the IDI to be a useful measure, bootstrap methods may offer a reasonable option for inference when evaluating new predictors, as long as the added predictive capacity is large.Keywords
This publication has 16 references indexed in Scilit:
- Recent advances in evaluating the prognostic value of a markerScandinavian Journal of Clinical and Laboratory Investigation, 2010
- Integrated discrimination improvement and probability‐sensitive AUC variantsStatistics in Medicine, 2009
- Postprandial Glucose Improves the Risk Prediction of Cardiovascular Death Beyond the Metabolic Syndrome in the Nondiabetic PopulationDiabetes Care, 2009
- Criteria for Evaluation of Novel Markers of Cardiovascular RiskCirculation, 2009
- Measures to Summarize and Compare the Predictive Capacity of MarkersThe International Journal of Biostatistics, 2009
- Gauging the Performance of SNPs, Biomarkers, and Clinical Factors for Predicting Risk of Breast CancerJNCI Journal of the National Cancer Institute, 2008
- Comments on ‘Evaluating the added predictive ability of a new marker: From area under the ROC curve to reclassification and beyond’ by M. J. Pencina et al., Statistics in Medicine (DOI: 10.1002/sim.2929)Statistics in Medicine, 2007
- Evaluating the added predictive ability of a new marker: From area under the ROC curve to reclassification and beyondStatistics in Medicine, 2007
- The need for reorientation toward cost-effective prediction: Comments on ‘Evaluating the added predictive ability of a new marker: From area under the ROC curve to reclassification and beyond’ by Pencinaet al.,Statistics in Medicine (DOI: 10.1002/sim.2929)Statistics in Medicine, 2007
- A phase III clinical trial of antibiotics to reduce chorioamnionitis-related perinatal HIV-1 transmissionAIDS, 2006