How Online Learning Approaches Ornstein Uhlenbeck Processes

We show that under reasonable conditions, online learning for a nonlinear function near a local minimum is similar to a multivariate Ornstein Uhlenbeck process. This implies that the parameter state oscillates randomly around the minimum point, with a Gaussian limiting distribution.

• Type: Journal
• Year: 2006
• Where: NPL
• Authors: Fredrik A. Dahl
• Keywords: Gaussian Limiting Distribution, Local Minimum, Multivariate Ornstein Uhlenbeck, NPL 2006