Confounding among Measures of Leniency and Halo
- 1 March 1989
- journal article
- research article
- Published by SAGE Publications in Educational and Psychological Measurement
- Vol. 49 (1), 1-10
- https://doi.org/10.1177/0013164489491001
Abstract
Because the variability of rating distributions decreases with leniency, common indices of the psychometric performance of rating data may be confounded. Lenient rating distributions, for example, should show lower average scale intercorrelations (a measure of halo) than do less lenient rating distributions. Although psychometric theory would predict such confounding, it does not permit the estimation of the magnitude of this phenomenon. Consequently, the interrelationships among halo and leniency measures were examined using simulated rating data. Specifically, effects of leniency on 3 levels of halo were examined. As expected, results showed that as leniency increased, halo decreased when measured by dimension intercorrelations but increased when measured by standard deviations across dimensions. The magnitude of the convergent validities calculated for halo and leniency measures also decreased as leniency increased. In addition, although halo and leniency were orthogonal when no leniency was present, substantial confounding of these constructs occurred as leniency increased. Implications of these results for the use and integration of the various measures are discussed.Keywords
This publication has 7 references indexed in Scilit:
- A method for correcting parameter estimates in samples subject to a ceiling.Psychological Bulletin, 1988
- A warning about the use of a standard deviation across dimensions within ratees to measure halo.Journal of Applied Psychology, 1986
- Ubiquitous halo.Psychological Bulletin, 1981
- Rating the ratings: Assessing the psychometric quality of rating data.Psychological Bulletin, 1980
- Effects of rater training: Creating new response sets and decreasing accuracy.Journal of Applied Psychology, 1980
- The mixed standard rating scale: An evaluationOrganizational Behavior and Human Performance, 1977
- Behavioral expectation scales: Effects of developmental procedures and formats.Journal of Applied Psychology, 1976