Law of the Iterated Logarithm for a Constant-Gain Linear Stochastic Gradient Algorithm

We study almost-sure limiting properties, taken as 0, of the finite horizon sequence of random estimates { 0, 1, 2, . . . , T/ } for the linear stochastic gradient algorithm n+1 = n + an+1 - ( n) Xn+1 Xn+1, 0 = nonrandom, where T (0, ) is an arbitrary constant, (0, 1] is a (small) adaptation gain, and {an} and {Xn} are data sequences which drive the algorithm. These limiting properties are expressed in the form of a functional law of the iterated logarithm. Key words. stochastic gradient algorithm, L-mixing processes, functional law of the iterated logarithm AMS subject classifications. 60F15, 60F17, 93E10 PII. S0363012997331007

• Type: Journal
• Year: 2000
• Where: SIAMCO
• Authors: J. A. Joslin, A. J. Heunis
• Keywords: Finite Horizon Sequence, SIAMCO 2000, Stochastic Gradient, Stochastic Gradient Algorithm