Improving the Learning Speed of 2-Layer Neural Networks by Choosing Initial Values of the Adaptive Weights
Venue
IJCNN International Joint Conference on Neural Networks, pp. 21-26 vol.3
Publication Year
Keywords
2-layer neural networks,adaptive systems,adaptive weights,complicated desired response,hidden units,initial weights,learning speed,learning systems,neural nets,nonlinear function,piecewise linear segments,training problems,training time,truck-backer-upper,two-layer neural network
Identifiers
Authors
- Derrick Nguyen
- Bernard Widrow
Abstract
The authors describe how a two-layer neural network can approximate any nonlinear function by forming a union of piecewise linear segments. A method is given for picking initial weights for the network to decrease training time. The authors have used the method to initialize adaptive weights over a large number of different training problems and have achieved major improvements in learning speed in every case. The improvement is best when a large number of hidden units is used with a complicated desired response. The authors have used the method to train the truck-backer-upper and were able to decrease the training time from about two days to four hours