Entropy reduction in Bayesian analysis of measurements

Abstract

Baye's theorem, when applied to the interpretation of a set of measurements with assumed multivariate Gaussian a priori distributions of the physical quantities to be determined, leads to explicit formulas for the most likely interpretation and the corresponding reduction in entropy. We point out an interesting complementarity in two equivalent expressions for the entropy decrease, one of which involves the determinant of a matrix connecting points in physical space, while the other involves the determinant of a matrix connecting the individual measurements. Off-diagonal elements of the latter matrix indicate the degree of redundancy of the information obtained from the corresponding pairs of measurements. Techniques for evaluating the Bayesian expressions for the most likely interpretation are briefly outlined.