Test-Retest Reliabilities of Two Treatment-preference Instruments in Measuring Utilities

Abstract

The authors assessed the test-retest reliabilities of two treatment-preference instruments recently applied to the measurement of the utilities of health states after different treatment modalities for cancer. The first instrument measures the strengths of preferences concerning a choice between a wait-and-see policy, and treatment with radiotherapy after an initial surgical breast-conserving procedure for early breast cancer. The second measures the strengths of preferences concerning a choice between two hypothetical surgical treatment outcomes in cancer of the rectum with different probabilities of expected five-year survival. Both measure the strength of a subject's treatment preference given probabilities of treat ment-related costs and benefits. The subjects were radiotherapy technicians (n = 20) and cancer patients (n = 20) who were interviewed in weeks 2 and 4 of radiotherapy. The test- retest reliabilities of both instruments were inconsistent and moderately high, with Spear- man's rank correlations ranging from 0.38 to 0.81 and weighted kappas ranging from 0.38 to 0.69. To investigate whether the start of treatment with radiotherapy influenced the utilities that patients assigned to health states, the same procedure was applied in another, com parable, group of patients with cancer ( n = 20). For this group, the first assessment was made prior to the start of treatment and the second during the second week of radiation therapy. The scores of this group of patients indeed appeared to be less stable than the scores of the patients assessed in weeks 2 and 4 of radiotherapy. However, the instability of the scores could have been the result of test bias. Key words: test-retest reliability; utility assessment; oncology. (Med Decis Making 1993;13:133-140)