Least Squares Estimation Without Priors or Supervision

Selection of an optimal estimator typically relies on either supervised training samples (pairs of measurements and their associated true values), or a prior probability model for the true values. Here, we consider the problem of obtaining a least-squares estimator given a measurement process with known statistics (i.e., a likelihood function), and a set of unsupervised measurements, each arising from a corresponding true value drawn randomly from an unknown distribution. We develop a general expression for a nonparametric empirical Bayes least squares (NEBLS) estimator, that expresses the optimal least-squares estimator in terms of the measurement density, with no explicit reference to the unknown (prior) density. We study the conditions under which such estimators exist, and derive specific forms for a variety of different measurement processes. We further show that each of these NEBLS estimators may be used to express the mean squared estimation error as an expectation over the me...

• Type: Journal
• Year: 2011
• Where: NECO
• Authors: Martin Raphan, Eero P. Simoncelli
• Keywords: Computer Networks, Estimators, Measurement, NECO 2011, True Value