The Answer Key as a Source of Error in Examinations for Professionals
- 1 December 1987
- journal article
- Published by Wiley in Journal of Educational Measurement
- Vol. 24 (4), 321-331
- https://doi.org/10.1111/j.1745-3984.1987.tb00283.x
Abstract
There has been a growing emphasis on written examinations that assess the ability of physicians and teachers to make decisions in the absence of protocols for action—a crucial aspect of professional competence. A characteristic of such tests is controversy, even among experts, about what constitutes the correct response to some of the items. This paper studied the impact of variability in answer keys, constructed using the aggregate method, on total errors of measurement. Results indicated that several scorers made a sizable contribution to reduction in measurement error and that scorers or groups of scorers who each developed the answer key for a subset of items produced better results than a single group that developed the answer key for all items. Implications of judgment tests scored using the aggregate method are described for teacher and physician certification.Keywords
This publication has 8 references indexed in Scilit:
- A Survey of State Teacher‐Competency Examination ProgramsEducational Measurement: Issues and Practice, 1987
- A National Teacher ExaminationEducational Measurement: Issues and Practice, 1985
- Objective measurement of clinical performanceMedical Education, 1985
- A Comparison of Knowledge, Synthesis, and Clinical JudgmentEvaluation & the Health Professions, 1984
- Scoring patient management problems: external validation of expert consensus.Evaluation & the Health Professions, 1982
- The Toss-upNew England Journal of Medicine, 1981
- Clinical Judgment: Psychological Research and Medical PracticeScience, 1976
- A Long-Term Project in Psychology: Lives Through Time . Jack Block. In collaboration with Norma Haan. Bancroft, Berkeley, Calif., 1971. xxii, 314 pp., illus. $12.50.Science, 1972