On Unique Decodability

In this paper, we propose a revisitation of the topic of unique decodability and of some fundamental theorems of lossless coding. It is widely believed that, for any discrete source X, every "uniquely decodable" block code satisfies E[l(X1;X2;. . . ;Xn)] H(X1;X2;. . . ;Xn) where X1;X2;. . . ;Xn are the first n symbols of the source, E[l(X1;X2;. . . ;Xn)] is the expected length of the code for those symbols, and H(X1;X2;. . . ;Xn) is their joint entropy. We show that, for certain sources with memory, the above inequality only holds when a limiting definition of "uniquely decodable code" is considered. In particular, the above inequality is usually assumed to hold for any "practical code" due to a debatable application of McMillan's theorem to sources with memory. We thus propose a clarification of the topic, also providing an extended version of McMillan's theorem to be used for Markovian sources.

• Type: Journal
• Year: 2008
• Where: CORR
• Authors: Marco Dalai, Riccardo Leonardi
• Keywords: Block Code Satisfies, Code, CORR 2008, Education, Fundamental Theorems