Multiple choice and true/false tests: reliability measures and some implications of negative marking
- 1 October 2004
- journal article
- research article
- Published by Taylor & Francis in Assessment & Evaluation in Higher Education
- Vol. 29 (5), 585-595
- https://doi.org/10.1080/02602930410001689153
Abstract
The standard error of measurement usefully provides confidence limits for scores in a given test, but is it possible to quantify the reliability of a test with just a single number that allows comparison of tests of different format? Reliability coefficients do not do this, being dependent on the spread of examinee attainment. Better in this regard is a measure produced by dividing the standard error of measurement by the test's ‘reliability length’, the latter defined as the maximum possible score minus the most probable score obtainable by blind guessing alone. This, however, can be unsatisfactory with negative marking (formula scoring), as shown by data on 13 negatively marked true/false tests. In these the examinees displayed considerable misinformation, which correlated negatively with correct knowledge. Negative marking can improve test reliability by penalizing such misinformation as well as by discouraging guessing. Reliability measures can be based on idealized theoretical models instead of on test data. These do not reflect the qualities of the test items, but can be focused on specific test objectives (e.g. in relation to cut‐off scores) and can be expressed as easily communicated statements even before tests are written.Keywords
This publication has 17 references indexed in Scilit:
- Misinformation, partial knowledge and guessing in true/false testsMedical Education, 2002
- Prospect theory analysis of guessing in multiple choice testsJournal of Behavioral Decision Making, 2002
- Do Item-discrimination Indices Really Help Us to Improve Our Tests?Assessment & Evaluation in Higher Education, 2001
- Quantifying the Effects of Chance in Multiple Choice and True/False Tests: Question selection and guessing of answersAssessment & Evaluation in Higher Education, 2001
- Statistical Modelling of Multiple?choice and True/False Tests: ways of considering, and of reducing, the uncertainties attributable to guessingAssessment & Evaluation in Higher Education, 1999
- Assessment of competence in technical clinical skills of general practitionersMedical Education, 1995
- A STUDY OF HYPOTHESES BASIC TO THE USE OF RIGHTS AND FORMULA SCORESJournal of Educational Measurement, 1984
- Multiple choice questions: to guess or not to guessMedical Education, 1976
- BLIND GUESSING ON OBJECTIVE ACHIEVEMENT TESTS1Journal of Educational Measurement, 1968
- Coefficient alpha and the internal structure of testsPsychometrika, 1951