Empirical Bayes Ranking Methods

Abstract

Ranking problems arise in setting priorities for investigations, in providing a simple summary of performance, in comparing objects in a manner robust to measurement scale, and in a wide variety of other applications. Commonly, rankings are computed from measurements that depend on the true attribute. Using the Gaussian model, we propose and compare methods for using these measurements to estimate the ranks of the underlying attributes and show that those based on an empirical Bayes model produce estimates that differ from ranking observed data. These differences result both from the effect of shrinking posterior means towards a common value by an amount that depends on the precision of individual measurements and from the Bayes processing of the posterior distribution to produce estimates that account for the uncertainty in the distribution of the ranks. We illustrate different ranking methods using data on school achievement reported by Aitkin and Longford (1986) . Mathematical and empirical results highlight the importance of using appropriate ranking methods and identify issues requiring further research.