A comparative method based on ROC analysis for the quantitation of observer performance in scintigraphy

Abstract

A method is proposed for the evaluation of observer performance based on the concept of count density: it provides a description of the effects of different display or processing techniques that is easily graphed. A set of ROC curves is produced for use as a calibration for other ROC graphs. Each curve is obtained experimentally using the same series of images and varying only the count density. The effect of an image processing operation is evaluated by comparison with the calibration curves, and is expressed as the change in count density in unprocessed images that produces the same effect on an observer as the processing. The relative change in count density is defined as the gain of the processing operation. The method is used to quantitate the effects of three linear filters, and of their respective noise textures, on the detectability of spheres 3 cm in diameter immersed in a uniform background. The smoothing filters gave the largest gain (2.5) of the three. When simulated image data were manipulated so that filtering changed the noise texture only, with spatial resolution and noise variance held constant, the smoothing filter provided a gain of about 0.25 for the targets considered; this corresponds to a loss of about 75% of the counts in an unprocessed image.