The Predictive Validity of Three Versions of the MCAT in Relation to Performance in Medical School, Residency, and Licensing Examinations: A Longitudinal Study of 36 Classes of Jefferson Medical College
- 1 June 2010
- journal article
- mcat
- Published by Wolters Kluwer Health in Academic Medicine
- Vol. 85 (6), 980-987
- https://doi.org/10.1097/acm.0b013e3181cece3d
Abstract
Purpose The Medical College Admission Test (MCAT) has undergone several revisions for content and validity since its inception. With another comprehensive review pending, this study examines changes in the predictive validity of the MCAT's three recent versions. Method Study participants were 7,859 matriculants in 36 classes entering Jefferson Medical College between 1970 and 2005; 1,728 took the pre-1978 version of the MCAT; 3,032 took the 1978–1991 version, and 3,099 took the post-1991 version. MCAT subtest scores were the predictors, and performance in medical school, attrition, scores on the medical licensing examinations, and ratings of clinical competence in the first year of residency were the criterion measures. Results No significant improvement in validity coefficients was observed for performance in medical school or residency. Validity coefficients for all three versions of the MCAT in predicting Part I/Step 1 remained stable (in the mid-0.40s, P < .01). A systematic decline was observed in the validity coefficients of the MCAT versions in predicting Part II/Step 2. It started at 0.47 for the pre-1978 version, decreased to between 0.42 and 0.40 for the 1978–1991 versions, and to 0.37 for the post-1991 version. Validity coefficients for the MCAT versions in predicting Part III/Step 3 remained near 0.30. These were generally larger for women than men. Conclusions Although the findings support the short- and long-term predictive validity of the MCAT, opportunities to strengthen it remain. Subsequent revisions should increase the test's ability to predict performance on United States Medical Licensing Examination Step 2 and must minimize the differential validity for gender.Keywords
This publication has 24 references indexed in Scilit:
- Components of postgraduate competence: analyses of thirty years of longitudinal dataMedical Education, 2007
- An empirical study of the predictive validity of number grades in medical school using 3 decades of longitudinal data: implications for a grading systemMedical Education, 2004
- Prediction of Studentsʼ Performances on Licensing Examinations Using Age, Race, Sex, Undergraduate GPAs, and MCAT ScoresAcademic Medicine, 2000
- Preliminary study of the accuracies of the old and new medical college admission tests for predicting performance on USMLE Step 1Academic Medicine, 1996
- Assessing the validity of the updated Medical College Admission TestAcademic Medicine, 1994
- A preliminary study of the validity of scores and pass/fail standards for USMLE steps 1 and 2Academic Medicine, 1993
- Correlating studentsʼ undergraduate science GPAs, their MCAT scores, and the academic caliber of their undergraduate colleges with their first-year academic performances across five classes at Dartmouth Medical SchoolAcademic Medicine, 1992
- Testing for validity and bias in the use of GPA and the MCAT in the selection of medical school students [published erratum appears in Acad Med 1991 Jan;66(1):25]Academic Medicine, 1990
- Using a standardized-patient examination to establish the predictive validity of the MCAT and undergraduate GPA as admissions criteriaAcademic Medicine, 1989
- Components of Clinical Competence Ratings of Physicians: An Empirical ApproachEducational and Psychological Measurement, 1986