Bayesian Correlation Score: A Utilitarian Measure of Forecast Skill

Abstract

From the theory of sufficient comparisons of experiments, a measure of skill is derived for categorical forecasts of continuous predictands. Called Bayesian correlation wore (BCS), the measure is specified in terms of three parameters of a normal-linear statistical model that combines information from two sources: a prior (climatological) record of the predictand and a verification record of forecasts. Three properties characterize the BCS: (i) It is meaningful for comparing alternative forecasts of the same predictand, as well as forecasts of different predictands, though in a limited sense; (ii) it is interpretable as correlation between the forecast and the predictand; and, most significantly, (iii) it orders alternative forecast systems consistently with their ex ante economic values to rational users (those who make decisions by maximizing the expected utility of outcomes under the posterior distribution of the predictand). Thus, by maximizing the BCS, forecasters can assure a utilitarian society of the maximum potential economic benefits of their forecast.