New method based on determining error surface for designing three layer neural networks

A method is proposed for designing three layer neural networks that assures global minimization of errors. The commonly used gradient-based learning algorithm suffers form the local minima problem, however, it can be solved if the error surface becomes convex. In the paper a number of possible network structures are provided together with their gradient-based learning algorithms. For a given set of training data, an appropriate network structure, i.e. the number of hidden nodes, the types of activation function, and the connections between them, is determined. All of the proposed structures give convex error surfaces and thus solve the local minima problem. The difference between them is in the level of locality and generalization ability. A numerical example is provided that supports the present approach.