Dissipation of Information in Channels With Input Constraints

8 years 6 months ago

Download people.lids.mit.edu

One of the basic tenets in information theory, the data processing inequality states that output divergence does not exceed the input divergence for any channel. For channels without input constraints, various estimates on the amount of such contraction are known, Dobrushin’s coeﬃcient for the total variation being perhaps the most well-known. This work investigates channels with average input cost constraint. It is found that while the contraction coeﬃcient typically equals one (no contraction), the information nevertheless dissipates. A certain nonlinear function, the Dobrushin curve of the channel, is proposed to quantify the amount of dissipation. Tools for evaluating the Dobrushin curve of additive-noise channels are developed based on coupling arguments. Some basic applications in stochastic control, uniqueness of Gibbs measures and fundamental limits of noisy circuits are discussed. As an application, it shown that in the chain of n power-constrained relays and Gaussian c...

Yury Polyanskiy, Yihong Wu 0001

Real-time Traffic