Observer variability in grading patients with subarachnoid hemorrhage

Abstract

The management of individual patients with subarachnoid hemorrhage depends greatly on assessment of the patient's clinical condition. Difficulty in applying current grading systems prompted the authors to conduct studies of observer variability and to attempt to identify sources of inconsistency. Observers graded 15 patients by both the Hunt and Hess and Nishioka systems. Considerable observer variability was found, with up to four different grades being selected for the same patient. Kappa statistics were used to evaluate the data. This method determines observer agreement occurring in excess of chance. Kappa values for each grading system showed observer agreement to be significantly better than chance, yet revealed marked observer variation. Most variation occurred when Grade 3 was selected, irrespective of the system used. In a further study where observers graded clinical summaries, similar variation occurred; therefore, inconsistency was due mainly to difficulty in matching patients with levels described in the grading system, rather than to fluctuation in the patient's clinical condition or difference in the observers' examination technique. Variability was high when patients with systemic disease or vasospasm on angiography were graded with the Hunt and Hess system. The studies show that a simpler and more reliable grading system is required, and emphasize and need for caution when interpreting the results from different published series.