Least-Squares Independent Component Analysis

Accurately evaluating statistical independence among random variables is a key element of Independent Component Analysis (ICA). In this paper, we employ a squared-loss variant of mutual information as an independence measure and give its estimation method. Our basic idea is to estimate the ratio of probability densities directly without going through density estimation, by which a hard task of density estimation can be avoided. In this density ratio approach, a natural cross-validation procedure is available for hyper-parameter selection. Thus, all tuning parameters such as the kernel width or the regularization parameter can be objectively optimized. This is an advantage over recently developed kernel-based independence measures and is a highly useful property in unsupervised learning problems such as ICA. Based on this novel independence measure, we develop an ICA algorithm named Least-squares Independent Component Analysis (LICA).

• Type: Journal
• Year: 2011
• Where: NECO
• Authors: Taiji Suzuki, Masashi Sugiyama
• Keywords: Computer Networks, Density Estimation, Independence Measure, Independent Component Analysis, NECO 2011