An adaptation of item modeling for developing test‐item banks

Abstract

An adaptation of item modeling, an item‐writing procedure, is described. As with related methods, the procedure includes dividing the stem of an item into discrete elements of information and then altering the elements to create new items. In this adaptation, writers were instructed to change the elements so that each incorrect option in the original item becomes the correct answer to a new item. Another part of the process, an innovation that increases the number of items generated, is to change the question asked of examinees (e.g., from diagnosis to treatment) for each item and write new response options. Two novice item writers applied modeling to patient‐based, one‐best‐answer items. They found the procedure easy to use and intuitive. From 13 items, they produced 209 modeled items in 24 at‐home hours, plus 12 hr of meeting time. A consultant evaluated a subset of items and estimated that approximately half the items were of sufficient quality to appear on a subspecialty certifying examination. Though the project had several limitations, item modeling appears to have promise as an efficient item‐writing procedure. Conservative estimates suggest it is about four times more productive than a traditional committee‐based approach.