Accounting for Observational Uncertainty in Forecast Verification: An Information-Theoretical View on Forecasts, Observations, and Truth
- 1 July 2011
- journal article
- Published by American Meteorological Society in Monthly Weather Review
- Vol. 139 (7), 2156-2162
- https://doi.org/10.1175/2011mwr3573.1
Abstract
Recently, an information-theoretical decomposition of Kullback–Leibler divergence into uncertainty, reliability, and resolution was introduced. In this article, this decomposition is generalized to the case where the observation is uncertain. Along with a modified decomposition of the divergence score, a second measure, the cross-entropy score, is presented, which measures the estimated information loss with respect to the truth instead of relative to the uncertain observations. The difference between the two scores is equal to the average observational uncertainty and vanishes when observations are assumed to be perfect. Not acknowledging for observation uncertainty can lead to both overestimation and underestimation of forecast skill, depending on the nature of the noise process.Keywords
This publication has 14 references indexed in Scilit:
- Information gain as a score for probabilistic forecastsMeteorlogical Applications, 2011
- Scoring Rules for Forecast VerificationMonthly Weather Review, 2010
- Reliability, sufficiency, and the decomposition of proper scoresQuarterly Journal of the Royal Meteorological Society, 2009
- Scoring Probabilistic Forecasts: The Importance of Being ProperWeather and Forecasting, 2007
- Incorporating Misclassification Error in Skill AssessmentMonthly Weather Review, 2005
- Probability TheoryPublished by Cambridge University Press (CUP) ,2003
- A New Vector Partition of the Probability ScoreJournal of Applied Meteorology, 1973
- A new interpretation of information rateIEEE Transactions on Information Theory, 1956
- On Information and SufficiencyThe Annals of Mathematical Statistics, 1951
- VERIFICATION OF FORECASTS EXPRESSED IN TERMS OF PROBABILITYMonthly Weather Review, 1950