Experimental Study of Ergodic Learning Curve in Hidden Markov Models

A number of learning machines used in information science are not regular, but rather singular, because they are non-identifiable and their Fisher information matrices are singular. Even for singular learning machines, the learning theory was developed for the case in which training samples are independent. However, if training samples have timedependency, then learning theory is not yet established. In the present paper, we define an ergodic generalization error for a time-dependent sequence and study its behavior experimentally in hidden Markov models. The ergodic generalization error is clarified to be inversely proportional to the number of training samples, but the learning coefficient depends strongly on time-dependency.

• Type: Conference
• Year: 2008
• Where: ICONIP
• Authors: Masashi Matsumoto, Sumio Watanabe
• Keywords: Ergodic Generalization Error, ICONIP 2008, Information Technology, Learning Machines, Training Samples