A METHOD FOR COMBINING OCCURRENCE AND NONOCCURRENCE INTEROBSERVER AGREEMENT SCORES
- 1 December 1978
- journal article
- research article
- Published by Wiley in Journal of Applied Behavior Analysis
- Vol. 11 (4), 523-527
- https://doi.org/10.1901/jaba.1978.11-523
Abstract
Various statistics have been proposed as standard methods for calculating and reporting interobserver agreement scores. The advantages and disadvantages of each have been discussed in this journal recently but without resolution. A formula is presented that combines separate measures of occurrence and nonoccurrence percentages of agreement, with weight assigned to each measure, varying according to the observed rate of behavior. This formula, which is a modification of a formula proposed by Clement (1976), appears to reduce distortions due to “chance” agreement encountered with very high or low observed rates of behavior while maintaining the mathematical and conceptual simplicity of the conventional method for calculating occurrence and nonoccurrence agreement.Keywords
This publication has 7 references indexed in Scilit:
- A REVIEW OF THE OBSERVATIONAL DATA‐COLLECTION AND RELIABILITY PROCEDURES REPORTED IN THE JOURNAL OF APPLIED BEHAVIOR ANALYSISJournal of Applied Behavior Analysis, 1977
- OBSERVER AGREEMENT, CREDIBILITY, AND JUDGMENT: SOME CONSIDERATIONS IN PRESENTING OBSERVER AGREEMENT DATAJournal of Applied Behavior Analysis, 1977
- EVALUATING INTEROBSERVER RELIABILITY OF INTERVAL DATA1Journal of Applied Behavior Analysis, 1977
- REVIEWER'S COMMENT: JUST BECAUSE IT'S RELIABLE DOESN'T MEAN THAT YOU CAN USE ITJournal of Applied Behavior Analysis, 1977
- CONSIDERATIONS IN THE CHOICE OF INTEROBSERVER RELIABILITY ESTIMATESJournal of Applied Behavior Analysis, 1977
- A PROBABILITY‐BASED FORMULA FOR CALCULATING INTEROBSERVER AGREEMENTJournal of Applied Behavior Analysis, 1977
- A Formula for Computing Inter-Observer AgreementPsychological Reports, 1976