Utility Theory Applied to Multivariable System Effectiveness Evaluation

Abstract

In this paper, utility and decision theories are applied to determine the effectiveness of a complex system with several measures of performance. A significant number of individuals, both systems analysts and operational personnel in the NADC and related organizations, were asked to rank the measures of performance according to their importance and to assign utility indices to each. Churchman and Ackoff's method for checking the consistency of the ranking has been modified and applied to the study. The evaluations of the individual judges were then adjusted, and the estimates of all judges were combined to form an overall utility index (measure of performance). The “Coefficient of Concordance” test was employed to measure the degree of agreement among the judges. The weighted overall measure of performance is used to evaluate proposed courses of action and to establish stopping rules for a simulation study.