Theory‐guided design of a rating scale for course evaluation in problem‐based curricula

Abstract

In this article, the development and evaluation of a rating scale for course improvement is described. Unlike other attempts in this area, design was guided by an explicit theory describing the teaching and learning processes taking place in problem‐based curricula. Data were collected in two curricula using the problem‐based learning approach. A confirmatory factor analysis was conducted to test to what extent the model underlying the rating scale described the data. In addition, generalizability and utility studies were conducted. The results indicate that the rating scale data fitted the underlying model reasonably well. Most factors of the model showed good generalizability, not only across facets judged but also across items and judges. The potential utility of the instrument was illustrated by analyses of some aspects of one of the curricula. It was argued that, although the instrument developed is useful only within the context of problem‐based curricula, the general approach followed here may be of use in every educational institution interested in program evaluation through systematic questioning of student‐observers.