Predictive validity of the multiple mini-interview for selecting medical trainees
- 20 July 2009
- journal article
- Published by Wiley in Medical Education
- Vol. 43 (8), 767-775
- https://doi.org/10.1111/j.1365-2923.2009.03407.x
Abstract
Introduction In this paper we report on further tests of the validity of the multiple mini-interview (MMI) selection process, comparing MMI scores with those achieved on a national high-stakes clinical skills examination. We also continue to explore the stability of candidate performance and the extent to which so-called ‘cognitive’ and ‘non-cognitive’ qualities should be deemed independent of one another. Methods To examine predictive validity, MMI data were matched with licensing examination data for both undergraduate (n = 34) and postgraduate (n = 22) samples of participants. To assess the stability of candidate performance, reliability coefficients were generated for eight distinct samples. Finally, correlations were calculated between ‘cognitive’ and ‘non-cognitive’ measures of ability collected in the admissions procedure, on graduation from medical school and 18 months into postgraduate training. Results The median reliability of eight administrations of the MMI in various cohorts was 0.73 when 12 10-minute stations were used with one examiner per station. The correlation between performance on the MMI and number of stations passed on an objective structured clinical examination-based licensing examination was r = 0.43 (P < 0.05) in a postgraduate sample and r = 0.35 (P < 0.05) in an undergraduate sample of subjects who sat the MMI 5 years prior to sitting the licensing examination. The correlation between ‘cognitive’ and ‘non-cognitive’ assessment instruments increased with time in training (i.e. as the focus of the assessments became more tailored to the clinical practice of medicine). Discussion Further evidence for the validity of the MMI approach to making admissions decisions has been provided. More generally, the reported findings cast further doubt on the extent to which performance can be captured with trait-based models of ability. Finally, although a complementary predictive relationship has consistently been observed between grade point average and MMI results, the extent to which cognitive and non-cognitive qualities are distinct appears to depend on the scope of practice within which the two classes of qualities are assessed.Keywords
This publication has 24 references indexed in Scilit:
- Factors affecting the utility of the multiple mini-interview in selecting candidates for graduate-entry medical schoolMedical Education, 2008
- Physician Scores on a National Clinical Skills Examination as Predictors of Complaints to Medical Regulatory AuthoritiesJAMA, 2007
- Uncertainties in the selection of applicants for medical schoolAdvances in Health Sciences Education, 2007
- Multiple mini-interviews predict clerkship and licensing examination performanceMedical Education, 2007
- Self-Discipline Outdoes IQ in Predicting Academic Performance of AdolescentsPsychological Science, 2005
- APPLIED RESEARCH: Reflecting the Relative Values of Community, Faculty, and Students in the Admissions Tools of Medical SchoolTeaching and Learning in Medicine, 2005
- On the generality of specificityMedical Education, 2003
- Assessing Personal Qualities in Medical School AdmissionsAcademic Medicine, 2003
- Global minimum essential requirements in medical educationMedical Teacher, 2002
- Knowledge and clinical problem-solvingMedical Education, 1985