Hierarchical maximum entropy density estimation

We study the problem of simultaneously estimating several densities where the datasets are organized into overlapping groups, such as a hierarchy. For this problem, we propose a maximum entropy formulation, which systematically incorporates the groups and allows us to share the strength of prediction across similar datasets. We derive general performance guarantees, and show how some previous approaches, such as hierarchical shrinkage and hierarchical priors, can be derived as special cases. We demonstrate the proposed technique on synthetic data and in a realworld application to modeling the geographic distributions of species hierarchically grouped in a taxonomy. Specifically, we model the geographic distributions of species in the Australian wet tropics and Northeast New South Wales. In these regions, small numbers of samples per species significantly hinder effective prediction. Substantial benefits are obtained by combining information across taxonomic groups.

• Type: Conference
• Year: 2007
• Where: ICML
• Authors: David M. Blei, Miroslav Dudík, Robert E. Schapire
• Keywords: Geographic Distributions, Hierarchical Priors, ICML 2007, Machine Learning, Maximum Entropy Formulation