caa a22 4500 475801024 CHVBK 20180406123737.0 cr unu---uuuuu 170329e20000201xx s 000 0 eng 10.1023/A:1009686825582 doi (NATIONALLICENCE)springer-10.1023/A:1009686825582 Towards the Optimal Learning Rate for Backpropagation [Elektronische Daten] [Danilo Mandic, Jonathon Chambers] A backpropagation learning algorithm for feedforward neural networks withan adaptive learning rate is derived. The algorithm is based uponminimising the instantaneous output error and does not include anysimplifications encountered in the corresponding Least Mean Square (LMS)algorithms for linear adaptive filters. The backpropagation algorithmwith an adaptive learning rate, which is derived based upon the Taylorseries expansion of the instantaneous output error, is shown to exhibitbehaviour similar to that of the Normalised LMS (NLMS) algorithm. Indeed,the derived optimal adaptive learning rate of a neural network trainedby backpropagation degenerates to the learning rate of the NLMS for a linear activation function of a neuron. By continuity, the optimal adaptive learning rate for neural networks imposes additional stabilisationeffects to the traditional backpropagation learning algorithm. Kluwer Academic Publishers, 2000 adaptive learning rate nationallicence backpropagation nationallicence feedforward neural networks nationallicence optimal gradient learning nationallicence Mandic Danilo School of Information Systems, University of East Anglia, NR4 7TJ, Norwich, UK aut Chambers Jonathon Dept. of Electrical and Electronic Engineering, Imperial College of Science, Technology and Medicine, Exhibition Road, SW7 2BT, London, UK aut Neural Processing Letters Kluwer Academic Publishers 11/1(2000-02-01), 1-5 1370-4621 11:1<1 2000 11 11063 https://doi.org/10.1023/A:1009686825582 text/html Onlinezugriff via DOI 1 research-article jats NATIONALLICENCE 856 40 https://doi.org/10.1023/A:1009686825582 text/html Onlinezugriff via DOI NATIONALLICENCE 700 1- Mandic Danilo School of Information Systems, University of East Anglia, NR4 7TJ, Norwich, UK aut NATIONALLICENCE 700 1- Chambers Jonathon Dept. of Electrical and Electronic Engineering, Imperial College of Science, Technology and Medicine, Exhibition Road, SW7 2BT, London, UK aut NATIONALLICENCE 773 0- Neural Processing Letters Kluwer Academic Publishers 11/1(2000-02-01), 1-5 1370-4621 11:1<1 2000 11 11063 Metadata rights reserved Springer special CC-BY-NC licence nationallicence BK010053 XK010053 XK010000 NATIONALLICENCE NATIONALLICENCE NL-springer