Agreement Among Physician Assessment Methods Searching for the Truth Among Fallible Methods

Abstract

To determine the convergent validity of four methods of physician assessment--physician interview, patient interview, chart audit, and videotaped observation--these methods were compared for their ability to detect medication regimens prescribed for patients with chronic obstructive pulmonary disease (COPD). Comparisons of data from the four methods revealed substantial discrepancies among them. In fact, the methods were in full accord only 36% of the time in detecting theophylline prescription, and even less often for the other COPD medications. According to physician interview, 78% of patients were on theophylline; chart audit revealed 62% of patients were on the medication; videotaped observation, 69%; and only 59% of patients reported themselves to be on theophylline. An iterative analysis, applied to determine which method most accurately captures data, revealed that reports from physician interviews were the most precise source of data. Although the order of merit was much the same for each of the drugs studied, there were some differences in levels of sensitivity across drugs. Specificities were consistently high for all drugs and all methods.