Issues and approaches to estimating interrater reliability in nursing research

Abstract

Following a general discussion of the meaning of and need for reliability estimates, four different approaches to the estimation of interrater reliability are discussed: correlational techniques, comparison of means, percentage of agreement, and generalizability theory techniques. Data from a nursing research project are used to illustrate the method and to interpret reliability estimates obtained by each approach. The estimates vary widely, from a low of .03 (using a percentage of agreement technique), to a high of .68 (using a correlational approach). The usefulness and advantages of generalizability theory techniques, which afford the most comprehensive and complex means of assessing reliability, are emphasized.