Estimating the number of components in a mixture of multilayer perceptrons

BIC criterion is widely used by the neural-network community for model selection tasks, although its convergence properties are not always theoretically established. In this paper we will focus on estimating the number of components in a mixture of multilayer perceptrons and proving the convergence of the BIC criterion in this frame. The penalized marginal-likelihood for mixture models and hidden Markov models introduced by Keribin (2000) and, respectively, Gassiat (2002) is extended to mixtures of multilayer perceptrons for which a penalized-likelihood criterion is proposed. We prove its convergence under some hypothesis which involve essentially the bracketing entropy of the generalized scorefunctions class and illustrate it by some numerical examples.

• Type: Journal
• Year: 2008
• Where: IJON
• Authors: Madalina Olteanu, Joseph Rynkiewicz
• Keywords: BIC Criterion, IJON 2006, IJON 2008, Model Selection Tasks, Multilayer Perceptrons