The Reliability and Validity of the American Board of Internal Medicine Monthly Evaluation Form
- 1 November 2003
- journal article
- research article
- Published by Wolters Kluwer Health in Academic Medicine
- Vol. 78 (11), 1175-1182
- https://doi.org/10.1097/00001888-200311000-00021
Abstract
To report indicators of reliability and validity of the American Board of Internal Medicine Evaluation Form (ABIM-MEF) at one institution (Wright-Patterson Medical Center). Completed ABIM-MEFs from 1990-1999 were reviewed. Reliability measures included Cronbach α, interrater reliability, and rating consistency between different types of staff and rotations. Construct validity was investigated by tracking ABIM-MEF scores over time and with factor analysis. Predictive validity was assessed by correlating ABIM-MEF scores with the In-training Examination and ABIM Certifying Examination results. The 71 residents averaged 12 ABIM-MEFs per year. The forms had a Cronbach α of 0.96 and high interrater reliability (intraclass correlation coefficients > 0.80). Ratings did not differ by type of attending or rotation, except that noninternists rated residents lower on procedural skills. ABIM-MEF questions about judgment, knowledge, and clinical skills showed significant improvement from month to month during each academic year as well as year to year. In contrast, questions on professional attitudes, humanism, and procedural skills sections improved between postgraduate year 1 and postgraduate year 2 only. ABIM-MEF questions collapsed into two domains in factor analysis: judgment-knowledge-skills and attitude-humanism. ABIM-MEF questions from judgment and knowledge sections modestly predicted In-training Examination and ABIM Certifying Examination results. In contrast, professional attitude, humanism, and clinical as well as procedural skill questions had little discriminative ability. The ABIM-MEF appears to be reliable and valid. Further, factor analysis results support the ABIM’s movement to simplify the monthly evaluation form to the new Accreditation Council for Graduate Medical Education core competencies.Keywords
This publication has 8 references indexed in Scilit:
- Methods for Evaluating the Clinical Competence of Residents in Internal Medicine: A ReviewAnnals of Internal Medicine, 1998
- The Mini-CEX (Clinical Evaluation Exercise): A Preliminary InvestigationAnnals of Internal Medicine, 1995
- Performance on the internal medicine second-year residency in-training examination predicts the outcome of the ABIM certifying examinationJournal of General Internal Medicine, 1994
- Do ratings on the american board of internal medicine resident evaluation form detect differences in clinical competence?Journal of General Internal Medicine, 1994
- Validity of the in-training examination for predicting american board of internal medicine certifying examination scoresJournal of General Internal Medicine, 1992
- The assessment of clinical skills/competence/performanceAcademic Medicine, 1990
- Evaluating evaluationJournal of General Internal Medicine, 1990
- The New Procedure for Evaluating the Clinical Competence of Candidates to Be Certified by the American Board of Internal MedicineAnnals of Internal Medicine, 1972