Abdul Hamid, Norhamreeza (2020) An improvement of back propagation algorithm using halley third order optimisation method for classification problems. Doctoral thesis, Universiti Tun Hussein Onn Malaysia.
|
Text
24p NORHAMREEZA ABDUL HAMID.pdf Download (16MB) | Preview |
|
![[img]](http://eprints.uthm.edu.my/style/images/fileicons/text.png) |
Text (Copyright Declaration)
NORHAMREEZA ABDUL HAMID COPYRIGHT DECLARATION.pdf Restricted to Repository staff only Download (19MB) | Request a copy |
|
|
Text (Full Text)
NORHAMREEZA ABDUL HAMID WATERMARK.pdf Restricted to Registered users only Download (20MB) | Request a copy |
Abstract
Back Propagation (BP) has proven to be a robust algorithm for different connectionist learning problems which commonly available for any functional induction that provides a computationally efficient method. This algorithm utilises first order optimisation method namely Gradient Descent (GD) method which attempts to minimise the error of network. Nevertheless, some major issues need to be considered. The GD method not performed well in large scale applications and when higher learning performances are required. Moreover, it has uncertainty in finding the global minimum of the error function. Besides, they generally depend on the parameters‟ selections. Thus, improving the BP learning efficiency has become an important area of research and consideration specifically in optimisation point of view. The variations of second order optimisation methods have been proposed which provide less iteration of convergence. Yet, an issue with these methods is occasionally converging to the undesired local minima. Inspired by the third order optimisation method which capable to solve unconstrained optimisation problems efficiently in the mathematical research area, this research endeavours to propose a new computational Halley method which is third order optimisation in improving the learning efficiency of BP algorithm namely Halley with Broyden-Fletcher-Goldfarb�Shanno (H-BFGS) and Halley with Davidon-Fletcher-Powell (H-DFP). The efficiency of the proposed methods is compared with the first and second order optimisation method by means of simulation on UCI Machine Learning Repository, Knowledge Extraction Evolutionary Learning and Kaggle dataset. The simulation results show that the highest improvement of H-BFGS in terms of generalisation accuracy is on the Voice Gender classification with 43.33% improvement for 60:40 data division. While H-DFP, the highest improvement achieved in generalisation accuracy is on the Seeds classification with 41.73% improvement for 70:30 data division. Thus, the proposed methods provide significant improvement and promising result in learning Artificial Neural Networks.
| Item Type: | Thesis (Doctoral) |
|---|---|
| Subjects: | Q Science > QA Mathematics > QA71-90 Instruments and machines > QA75-76.95 Calculating machines |
| Divisions: | Faculty of Computer Science and Information Technology > Department of Software Engineering |
| Depositing User: | Mrs. Sabarina Che Mat |
| Date Deposited: | 09 Sep 2021 04:04 |
| Last Modified: | 09 Sep 2021 04:04 |
| URI: | http://eprints.uthm.edu.my/id/eprint/923 |
Actions (login required)
 |
View Item |