Arbitrary source models and Bayesian codebooks in rate-distortion theory

-- We characterize the best achievable performance of lossy compression algorithms operating on arbitrary random sources, and with respect to general distortion measures. Direct and converse coding theorems are given for variable-rate codes operating at a fixed distortion level, emphasizing: (a) non-asymptotic results, (b) optimal or near-optimal redundancy bounds, and (c) results with probability one. This development is based in part on the observation that there is a precise correspondence between compression algorithms and probability measures on the reproduction alphabet. This is analogous to the Kraft inequality in lossless data compression. In the case of stationary ergodic sources our results reduce to the classical coding theorems. As an application of these general results, we examine the performance of codes based on mixture codebooks for discrete memoryless sources. A mixture codebook (or Bayesian codebook) is a random codebook generated from a mixture over some class of re...

• Type: Journal
• Year: 2002
• Where: TIT
• Authors: Ioannis Kontoyiannis, Junshan Zhang
• Keywords: Arbitrary Random Sources, Coding Theorems, Mixture Codebooks, TIT 2002