On the Consistency of Minimum Complexity Nonparametric Estimation

— Nonparametric estimation is usually inconsistent without some form of regularization. One way to impose regularity is through a prior measure. Barron and Cover [1], [2] have shown that complexitybased prior measures can insure consistency, at least when restricted to countable dense subsets of the infinite-dimensional parameter (i.e., function) space. Strangely, however, these results are independent of the actual complexity assignment: the same results hold under an arbitrary permutation of the match-up of complexities to functions. We will show that this phenomenon is related to the weakness of the convergence measures used. Stronger convergence can only be achieved through complexity measures that relate to the actual behavior of the functions.

• Type: Journal
• Year: 1998
• Where: TIT
• Authors: Zhiyi Chi, Stuart Geman
• Keywords: Actual Complexity Assignment, Nonparametric Estimation, Prior Measure, TIT 1998