Clinical trials and clinical practice: do doctors use the same criteria to judge outcome?

Abstract

The results of clinical trials often seem to have little influence on the practice of individual doctors. This could be because trial information is presented in the style of a scientific experiment which cannot often be clearly related to the context of everyday patient care. We tested the hypothesis that such framing effects would cause doctors to assess the clinical significance of treatment outcomes differently when presented as clinical trial results rather than as individual patient data. Fourteen rheumatologists independently reviewed the same 50 sets of data obtained from patients with rheumatoid arthritis. The data consisted of 10 commonly used clinical and laboratory variables measured before and after a period of treatment. The same data were presented in two formats on two separate occasions. The patient data format was a collection of typed sheets attributing each set of results to an individual patient. The clinical trial format was a professionally printed and bound booklet in which each set of results was laid out as summary results of a small uncontrolled clinical trial. Doctors judged the degree of improvement or deterioration and its clinical importance for each data set for both formats. These changes were converted into units of 'Clinical Importance'. Although some statistically significant differences emerged in the individual doctors' judgements between the formats none of these was of a clinically important size. The median of the mean trial--patient difference between the formats for all 14 doctors was 0.035 units of clinical importance [95% CI -0.244 to 0.074]. This evidence does not support the hypothesis that framing effects are a major cause of the failure of clinical trials to influence clinical practice.