Internship ratings as a validity outcome measure for an evaluation system to identify inadequate clerkship performance

Abstract

PURPOSE: To determine whether a third-year clerkship evaluation system validly predicted students' later performance ratings during their internships. METHOD: Adequacy of students' clerkship performances and the need for remediation were determined during seven consecutive academic years at the Uniformed Services University of the Health Sciences F. Edward Hebert School of Medicine using a criterion-based process including formal evaluation sessions. Internship ratings of students who needed remediation (remediators) from 1986 to 1993 were compared with those of students who voluntarily chose fourth-year medicine (non-remediators). Using written questionnaires, internship directors rated the graduates on a five-point scale for fund of knowledge, attitude, and analytic ability, and also provided written comments. RESULTS: Responses to questionnaires were available for 75 of 97 remediators (78%) and 268 of 313 non-remediators (86%). The remediators were 12.9 times more likely to have low internship performance scores and 9.4 times more likely to receive unfavorable comments than were the non-remediators. However, the majority of the remediators (80%) received only favorable comments. The medicine clerkship grade was more sensitive than the non-medicine grade-point average in predicting problems during internship (75% vs 8%). CONCLUSION: The medicine clerkship evaluation process detected whether a student was likely to have problems during internship, and the internship ratings supported the predictive validity of the evaluation system. The majority of students who were successfully remediated had no identifiable problems during internship.