Training Reformulated Product Units in Hybrid Neural Networks

— Higher order networks allow modelling of correlates and geometrically invariant properties. Current techniques for their development either require domain knowledge, or are constrained by scaling properties or local minima. A novel reformulation of the product unit is introduced, motivated by a desire to improve scaling and training properties. The new unit allows developing high orders of positive and negative powers, and correlates in a single stage, but can be trained successfully using standard back propagation techniques. Tests on standard benchmarks in various hybrid topologies demonstrate the potential in a variety of problem domains.

• Type: Conference
• Year: 2006
• Where: IJCNN
• Authors: Philip T. Elliott, Diven Topiwala, Will N. Browne
• Keywords: IJCNN 2006, Invariant Properties, Networks Allow Modelling, Various Hybrid Topologies