Natural gradient approach in orthogonal matrix optimization using cayley transform

Matrix optimization with orthogonal constraints appear in a variety of application fields including signal and image processing. Several researchers have developed algorithms for orthogonal matrix optimization using the Cayley transform that parameterizes the group of orthogonal matrices by the space of skew-symmetric matrices. However those algorithms sometimes have experienced extremely slow progress in their convergence. This paper introduces natural gradient approach to circumvent the slow progress. We show that, while the gradient algorithm based on the metric of the space of skew-symmetric matrices (“conventional gradient”) slows down when it comes close to the singular points of the Cayley transform, the gradient algorithm based on the metric of the group of orthogonal matrices (“natural gradient”) does not. We verify the result using a numerical simulation.

• Type: Journal
• Year: 2011
• Where: ICASSP
• Authors: Gen Hori
• Keywords: Gradient, ICASSP 2011, Orthogonal, Orthogonal Matrices, Signal Processing