The Use of Judgment Matrices in Subjective Workload Assessment: The Subjective Workload Dominance (SWORD) Technique

Abstract

One objective of the project was to determine compare two analytic algorithms for converting judgment matrices into subjective workload ratings. The original eigenvector algorithm used in Saaty's Analytic Hierarchy Process (AHP) was compared an algorithm of calculating geometric means. Also, three methods of identifying excessively inconsistent matrices were compared. Data from nine previous experiments were re-examined in the present analysis. There were no differences between the AHP ratings and the geometric mean ratings in terms of their sensitivity to the experimental manipulations. However, two of the inconsistency measures were successfully used to cull the data-sets of inconsistent matrices and improved the statistical sensitivity of one set of ratings. These findings suggest that: (1) the computationally simpler geometric means algorithm can be used as an alternative to the eigenvector algorithm, and (2) culling inconsistent matrices can sometimes improve rating sensitivity. These findings, along with previous research, demonstrate that judgment matrices can be a very valuable workload assessment tool. The essential steps for the proper use of judgment matrices in workload assessment are reviewed. A user's guide and software are also being prepared to aid researchers and practitioners