Shrinkage algorithms for MMSE covariance estimation

We address covariance estimation in the sense of minimum mean-squared error (MMSE) when the samples are Gaussian distributed. Specifically, we consider shrinkage methods which are suitable for high dimensional problems with a small number of samples (large p small n). First, we improve on the Ledoit-Wolf (LW) method by conditioning on a sufficient statistic. By the Rao-Blackwell theorem, this yields a new estimator called RBLW, whose mean-squared error dominates that of LW for Gaussian variables. Second, to further reduce the estimation error, we propose an iterative approach which approximates the clairvoyant shrinkage estimator. Convergence of this iterative method is established and a closed form expression for the limit is determined, which is referred to as the oracle approximating shrinkage (OAS) estimator. Both RBLW and OAS estimators have simple expressions and are easily implemented. Although the two methods are developed from different perspectives, their structure is identic...

• Type: Journal
• Year: 2010
• Where: TSP
• Authors: Yilun Chen, Ami Wiesel, Yonina C. Eldar, Alfred O. Hero
• Keywords: Artificial Intelligence, Clairvoyant Shrinkage Estimator, Estimator, Mean-squared Error, TSP 2010