On entropy for mixtures of discrete and continuous variables

Let X be a discrete random variable with support S and f : S S be a bijection. Then it is wellknown that the entropy of X is the same as the entropy of f(X). This entropy preservation property has been well-utilized to establish non-trivial properties of discrete stochastic processes, e.g. queuing process [1]. Entropy as well as entropy preservation is well-defined only in the context of purely discrete or continuous random variables. However for a mixture of discrete and continuous random variables, which arise in many interesting situations, the notions of entropy and entropy preservation have not been well understood. In this paper, we extend the notion of entropy in a natural manner for a mixed-pair random variable, a pair of random variables with one discrete and the other continuous. Our extensions are consistent with the existing definitions of entropy in the sense that there exist natural injections from discrete or continuous random variables into mixed-pair random variables ...

• Type: Journal
• Year: 2006
• Where: CORR
• Authors: Chandra Nair, Balaji Prabhakar, Devavrat Shah
• Keywords: CORR 2006, Education, Entropy, Entropy Preservation, Random Variables