Noiseless coding of correlated information sources

Abstract

Correlated information sequences\cdots ,X_{-1},X_0,X_1, \cdotsand\cdots,Y_{-1},Y_0,Y_1, \cdotsare generated by repeated independent drawings of a pair of discrete random variablesX, Yfrom a given bivariate distributionP_{XY} (x,y). We determine the minimum number of bits per characterR_XandR_Yneeded to encode these sequences so that they can be faithfully reproduced under a variety of assumptions regarding the encoders and decoders. The results, some of which are not at all obvious, are presented as an admissible rate region\mathcal{R}in theR_X - R_Yplane. They generalize a similar and well-known result for a single information sequence, namelyR_X \geq H (X)for faithful reproduction.