Improvements of back propagation algorithm performance by adaptively changing gain, momentum and learning rate

Abdul Hamid, Norhamreeza and Mohd Nawi, Nazri and Ghazali, Rozaida and Mohd Salleh, Mohd Najib Improvements of back propagation algorithm performance by adaptively changing gain, momentum and learning rate. International Journal of New Computer Architectures and Their Applications (IJNCAA), 1 (2). pp. 889-901. ISSN 2220-9085

Full text not available from this repository.

Abstract

In some practical Neural Network (NN) applications, fast response to external events within enormously short time is highly demanded. However, by using back propagation (BP) based on gradient descent optimization method obviously not satisfy in several application due to serious problems associate with BP which are slow learning convergence velocity and confinement to shallow minima. Over the years, many improvements and modification of the BP learning algorithm have been reported. In this research, we modified existing BP learning algorithm with adaptive gain by adaptively change the momentum coefficient and learning rate. In learning the patterns, the simulation results indicate that the proposed algorithm can hasten up the convergence behaviour as well as slide the network through shallow local minima compared to conventional BP algorithm. We use five common benchmark classification problems to illustrate the improvement of proposed algorithm.

Item Type:Article
Uncontrolled Keywords:back propagation; convergence speed; shallow minima; adaptive gain; adaptive momentum, adaptive learning rate
Subjects:Q Science > QA Mathematics > QA75 Calculating machines > QA75.5 Electronic computers. Computer science
Divisions:Faculty of Computer Science and Information Technology > Department of Software Engineering
ID Code:2964
Deposited By:Normajihan Abd. Rahman
Deposited On:07 Feb 2013 13:46
Last Modified:22 Jan 2015 11:49

Repository Staff Only: item control page