Spiculation is a primary sign of malignancy for masses detected by mammography. In this study, we developed a technique that analyzes patterns and quantifies the degree of spiculation present. Our current approach involves (1) automatic lesion extraction using region growing and (2) feature extraction using radial edge‐gradient analysis. Two spiculation measures are obtained from an analysis of radial edge gradients. These measures are evaluated in four different neighborhoods about the extracted mammographic mass. The performance of each of the two measures of spiculation was tested on a database of 95 mammographic masses using ROC analysis that evaluates their individual ability to determine the likelihood of malignancy of a mass. The dependence of the performance of these measures on the choice of neighborhood was analyzed. We have found that it is only necessary to accurately extract an approximate outline of a mass lesion for the purposes of this analysis since the choice of a neighborhood that accommodates the thin spicules at the margin allows for the assessment of margin spiculation with the radial edge‐gradient analysis technique. The two measures performed at their highest level when the surrounding periphery of the extracted region is used for feature extraction, yielding A z values of 0.83 and 0.85, respectively, for the determination of malignancy. These are similar to that achieved when a radiologist’s ratings of spiculation (A z =0.85) are used alone. The maximum value of one of the two spiculation measures (FWHM) from the four neighborhoods yielded an A z of 0.88 in the classification of mammographic mass lesions.