Examining the Effects of Differential Item (Functioning and Differential) Test Functioning on Selection Decisions: When Are Statistically Significant Effects Practically Important?
- 1 June 2004
- journal article
- Published by American Psychological Association (APA) in Journal of Applied Psychology
- Vol. 89 (3), 497-508
- https://doi.org/10.1037/0021-9010.89.3.497
Abstract
Item response theory differential test functioning (DTP) methods are often used to address issues in personnel selection, but the results are frequently difficult to interpret because statistically significant findings may have little practical importance. In this article, the authors proposed 2 effect size measures for DTP. One related DTP to mean raw score differences across groups: the other related DTP to the 4/5th rule for adverse impact at successive cut scores. The effects of DTP were examined in the context of personality assessment, professional licensure, and college admissions. Overall, the result indicated that although many items exhibited bias in analyses of the large samples, the net magnitudes of effect on potential selection decisions were nugatory.Keywords
This publication has 25 references indexed in Scilit:
- Effects of the testing situation on item responding: Cause for concern.Journal of Applied Psychology, 2001
- Detecting Faking on a Personality Instrument Using Appropriateness MeasurementApplied Psychological Measurement, 1996
- Within-group norming and other forms of score adjustment in preemployment testing.American Psychologist, 1994
- Significance Tests are Not EnoughTheory & Psychology, 1991
- A logistic mixture distribution model for polychotomous item responsesBritish Journal of Mathematical and Statistical Psychology, 1991
- Significance tests and the duplicity of binary decisions.Psychological Bulletin, 1989
- Study of the measurement bias of two standardized psychological tests.Journal of Applied Psychology, 1987
- Comparison of IRT True-Score and Equipercentile Observed-Score "Equatings"Applied Psychological Measurement, 1984
- Scrutinizing psychological tests: Measurement equivalence and equivalent relations with external variables are the central issues.Psychological Bulletin, 1984
- Biased test items and differential validity.Psychological Bulletin, 1982