Near optimal lossy source coding and compression-based denoising via Markov chain Monte Carlo

— We propose an implementable new universal lossy source coding algorithm. The new algorithm utilizes two wellknown tools from statistical physics and computer science: Gibbs sampling and simulated annealing. In order to code a source sequence xn , the encoder initializes the reconstruction block as ˆxn = xn , and then at each iteration uniformly at random chooses one of the symbols of ˆxn , and updates it. This updating is based on some conditional probability distribution which depends on a parameter β representing inverse temperature, an integer parameter k = o(log n) representing context length, and the original source sequence. At the end of this process, the encoder outputs the Lempel-Ziv description of ˆxn , which the decoder deciphers perfectly, and sets as its reconstruction. The complexity of the proposed algorithm in each iteration is linear in k and independent of n. We prove that, for any stationary ergodic source, the algorithm achieves the optimal rate-distortion p...

• Type: Conference
• Year: 2008
• Where: CISS
• Authors: Shirin Jalali, Tsachy Weissman
• Keywords: CISS 2008, Information Management, Source Coding Algorithm, Source Sequence, Source Sequence Xn