An Analysis of Overfitting in Particle Swarm Optimised Neural Networks
The phenomenon of overfitting, where a feed-forward neural network (FFNN) over trains on training data at the cost of generalisation accuracy is known to be speci c to the training algorithm used. This study investigates over tting within the context of particle swarm optimised (PSO) FFNNs. Two o...
Main Author: | |
---|---|
Other Authors: | |
Language: | en |
Published: |
University of Pretoria
2015
|
Subjects: | |
Online Access: | http://hdl.handle.net/2263/46273 van Wyk, AB 2014, An Analysis of Overfitting in Particle Swarm Optimised Neural Networks, MSc Dissertation, University of Pretoria, Pretoria, viewed yymmdd <http://hdl.handle.net/2263/46273> |
id |
ndltd-netd.ac.za-oai-union.ndltd.org-up-oai-repository.up.ac.za-2263-46273 |
---|---|
record_format |
oai_dc |
spelling |
ndltd-netd.ac.za-oai-union.ndltd.org-up-oai-repository.up.ac.za-2263-462732020-06-02T03:18:21Z An Analysis of Overfitting in Particle Swarm Optimised Neural Networks van Wyk, Andrich Benjamin Engelbrecht, Andries P. UCTD Particle swarm optimization (PSO) Feedforward neural networks Overfitting Adaptive neural networks The phenomenon of overfitting, where a feed-forward neural network (FFNN) over trains on training data at the cost of generalisation accuracy is known to be speci c to the training algorithm used. This study investigates over tting within the context of particle swarm optimised (PSO) FFNNs. Two of the most widely used PSO algorithms are compared in terms of FFNN accuracy and a description of the over tting behaviour is established. Each of the PSO components are in turn investigated to determine their e ect on FFNN over tting. A study of the maximum velocity (Vmax) parameter is performed and it is found that smaller Vmax values are optimal for FFNN training. The analysis is extended to the inertia and acceleration coe cient parameters, where it is shown that speci c interactions among the parameters have a dominant e ect on the resultant FFNN accuracy and may be used to reduce over tting. Further, the signi cant e ect of the swarm size on network accuracy is also shown, with a critical range being identi ed for the swarm size for e ective training. The study is concluded with an investigation into the e ect of the di erent activation functions. Given strong empirical evidence, an hypothesis is made that stating the gradient of the activation function signi cantly a ects the convergence of the PSO. Lastly, the PSO is shown to be a very effective algorithm for the training of self-adaptive FFNNs, capable of learning from unscaled data. Dissertation (MSc)--University of Pretoria, 2014. tm2015 Computer Science MSc Unrestricted 2015-07-02T11:08:33Z 2015-07-02T11:08:33Z 2015/04/21 2014 Dissertation http://hdl.handle.net/2263/46273 van Wyk, AB 2014, An Analysis of Overfitting in Particle Swarm Optimised Neural Networks, MSc Dissertation, University of Pretoria, Pretoria, viewed yymmdd <http://hdl.handle.net/2263/46273> A2015 4318323 en © 2015 University of Pretoria. All rights reserved. The copyright in this work vests in the University of Pretoria. No part of this work may be reproduced or transmitted in any form or by any means, without the prior written permission of the University of Pretoria. University of Pretoria |
collection |
NDLTD |
language |
en |
sources |
NDLTD |
topic |
UCTD Particle swarm optimization (PSO) Feedforward neural networks Overfitting Adaptive neural networks |
spellingShingle |
UCTD Particle swarm optimization (PSO) Feedforward neural networks Overfitting Adaptive neural networks van Wyk, Andrich Benjamin An Analysis of Overfitting in Particle Swarm Optimised Neural Networks |
description |
The phenomenon of overfitting, where a feed-forward neural network (FFNN) over trains
on training data at the cost of generalisation accuracy is known to be speci c to the
training algorithm used. This study investigates over tting within the context of particle
swarm optimised (PSO) FFNNs. Two of the most widely used PSO algorithms are
compared in terms of FFNN accuracy and a description of the over tting behaviour is
established. Each of the PSO components are in turn investigated to determine their
e ect on FFNN over tting. A study of the maximum velocity (Vmax) parameter is
performed and it is found that smaller Vmax values are optimal for FFNN training. The
analysis is extended to the inertia and acceleration coe cient parameters, where it is
shown that speci c interactions among the parameters have a dominant e ect on the
resultant FFNN accuracy and may be used to reduce over tting. Further, the signi cant
e ect of the swarm size on network accuracy is also shown, with a critical range being
identi ed for the swarm size for e ective training. The study is concluded with an
investigation into the e ect of the di erent activation functions. Given strong empirical
evidence, an hypothesis is made that stating the gradient of the activation function
signi cantly a ects the convergence of the PSO. Lastly, the PSO is shown to be a very
effective algorithm for the training of self-adaptive FFNNs, capable of learning from
unscaled data. === Dissertation (MSc)--University of Pretoria, 2014. === tm2015 === Computer Science === MSc === Unrestricted |
author2 |
Engelbrecht, Andries P. |
author_facet |
Engelbrecht, Andries P. van Wyk, Andrich Benjamin |
author |
van Wyk, Andrich Benjamin |
author_sort |
van Wyk, Andrich Benjamin |
title |
An Analysis of Overfitting in Particle Swarm Optimised Neural Networks |
title_short |
An Analysis of Overfitting in Particle Swarm Optimised Neural Networks |
title_full |
An Analysis of Overfitting in Particle Swarm Optimised Neural Networks |
title_fullStr |
An Analysis of Overfitting in Particle Swarm Optimised Neural Networks |
title_full_unstemmed |
An Analysis of Overfitting in Particle Swarm Optimised Neural Networks |
title_sort |
analysis of overfitting in particle swarm optimised neural networks |
publisher |
University of Pretoria |
publishDate |
2015 |
url |
http://hdl.handle.net/2263/46273 van Wyk, AB 2014, An Analysis of Overfitting in Particle Swarm Optimised Neural Networks, MSc Dissertation, University of Pretoria, Pretoria, viewed yymmdd <http://hdl.handle.net/2263/46273> |
work_keys_str_mv |
AT vanwykandrichbenjamin ananalysisofoverfittinginparticleswarmoptimisedneuralnetworks AT vanwykandrichbenjamin analysisofoverfittinginparticleswarmoptimisedneuralnetworks |
_version_ |
1719316465085054976 |