Rate distortion and denoising of individual data using Kolmogorov complexity

We examine the structure of families of distortion balls from the perspective of Kolmogorov complexity. Special attention is paid to the canonical rate-distortion function of a source word which returns the minimal Kolmogorov complexity of all distortion balls containing that word subject to a bound on their cardinality. This canonical ratedistortion function is related to the more standard algorithmic rate-distortion function for the given distortion measure. Examples are given of list distortion, Hamming distortion, and Euclidean distortion. The algorithmic rate-distortion function can behave differently from Shannon's rate-distortion function. To this end, we show that the canonical ratedistortion function can and does assume a wide class of shapes (unlike Shannon's); we relate low algorithmic mutual information to low Kolmogorov complexity (and consequently suggest that certain aspects of the mutual information formulation of Shannon's rate-distortion function behav...

• Type: Journal
• Year: 2010
• Where: TIT
• Authors: Nikolai K. Vereshchagin, Paul M. B. Vitányi
• Keywords: Distortion, Education, Kolmogorov Complexity, Rate-distortion Function, TIT 2010