Decision Models for Use with Criterion-Referenced Tests

Abstract

The problem of mastery decisions and optimizing cutoff scores on criterion-referenced tests is con sidered. This problem can be formalized as an (em pirical) Bayes problem with decisions rules of a monotone shape. Next, the derivation of optimal cutoff scores for threshold, linear, and normal ogive loss functions is addressed, alternately using such psychometric models as the classical model, the beta-binomial, and the bivariate normal model. One important distinction made is between deci sions with an internal and an external criterion. A natural solution to the problem of reliability and validity analysis of mastery decisions is to analyze with a standardization of the Bayes risk (coefficient delta). It is indicated how this analysis proceeds and how, in a number of cases, it leads to coeffi cients already known from classical test theory. Fi nally, some new lines of research are suggested along with other aspects of criterion-referenced test ing that can be approached from a decision-the oretic point of view.