Improving the Learning Speed of 2-Layer Neural Networks by Choosing Initial Values of the Adaptive Weights

Venue

IJCNN International Joint Conference on Neural Networks, pp. 21-26 vol.3

Publication Year

1990

Keywords

2-layer neural networks,adaptive systems,adaptive weights,complicated desired response,hidden units,initial weights,learning speed,learning systems,neural nets,nonlinear function,piecewise linear segments,training problems,training time,truck-backer-upper,two-layer neural network

Identifiers

Authors

  • Derrick Nguyen
  • Bernard Widrow

Abstract

The authors describe how a two-layer neural network can approximate any nonlinear function by forming a union of piecewise linear segments. A method is given for picking initial weights for the network to decrease training time. The authors have used the method to initialize adaptive weights over a large number of different training problems and have achieved major improvements in learning speed in every case. The improvement is best when a large number of hidden units is used with a complicated desired response. The authors have used the method to train the truck-backer-upper and were able to decrease the training time from about two days to four hours

Source Materials