Undersmoothed Kernel Entropy Estimators

We develop a "plug-in" kernel estimator for the differential entropy that is consistent even if the kernel width tends to zero as quickly as 1/N, where N is the number of i.i.d. samples. Thus, accurate density estimates are not required for accurate kernel entropy estimates; in fact, it is a good idea when estimating entropy to sacrifice some accuracy in the quality of the corresponding density estimate.

• Type: Journal
• Year: 2008
• Where: TIT
• Authors: Liam Paninski, Masanao Yajima
• Keywords: Accurate Kernel Entropy, Density Estimate, Entropy, TIT 2008