Optimization of CNN through Novel Training Strategy for Visual Classification Problems
The convolution neural network (CNN) has achieved state-of-the-art performance in many computer vision applications e.g., classification, recognition, detection, etc. However, the global optimization of CNN training is still a problem. Fast classification and training play a key role in the developm...
Main Authors: | , , , , , |
---|---|
Format: | Article |
Language: | English |
Published: |
MDPI AG
2018-04-01
|
Series: | Entropy |
Subjects: | |
Online Access: | http://www.mdpi.com/1099-4300/20/4/290 |
id |
doaj-c5898f72e95f49a3bc227ff7a007c007 |
---|---|
record_format |
Article |
spelling |
doaj-c5898f72e95f49a3bc227ff7a007c0072020-11-24T23:04:33ZengMDPI AGEntropy1099-43002018-04-0120429010.3390/e20040290e20040290Optimization of CNN through Novel Training Strategy for Visual Classification ProblemsSadaqat ur Rehman0Shanshan Tu1Obaid ur Rehman2Yongfeng Huang3Chathura M. Sarathchandra Magurawalage4Chin-Chen Chang5Department of Electronic Engineering, Tsinghua University, Beijing 100084, ChinaFaculty of Information Technology, Beijing University of Technology, Beijing 100022, ChinaDepartment of Electrical Engineering, Sarhad University of Science and IT, Peshawar 25000, PakistanDepartment of Electronic Engineering, Tsinghua University, Beijing 100084, ChinaSchool of Computer Science and Electronic Engineering, University of Essex, Colchester CO4 3SQ, UKDepartment of Information Engineering and Computer Science, Feng Chia University, Taichung City 407, TaiwanThe convolution neural network (CNN) has achieved state-of-the-art performance in many computer vision applications e.g., classification, recognition, detection, etc. However, the global optimization of CNN training is still a problem. Fast classification and training play a key role in the development of the CNN. We hypothesize that the smoother and optimized the training of a CNN goes, the more efficient the end result becomes. Therefore, in this paper, we implement a modified resilient backpropagation (MRPROP) algorithm to improve the convergence and efficiency of CNN training. Particularly, a tolerant band is introduced to avoid network overtraining, which is incorporated with the global best concept for weight updating criteria to allow the training algorithm of the CNN to optimize its weights more swiftly and precisely. For comparison, we present and analyze four different training algorithms for CNN along with MRPROP, i.e., resilient backpropagation (RPROP), Levenberg–Marquardt (LM), conjugate gradient (CG), and gradient descent with momentum (GDM). Experimental results showcase the merit of the proposed approach on a public face and skin dataset.http://www.mdpi.com/1099-4300/20/4/290CNN optimizationimage classificationMRPROPtraining algorithm |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Sadaqat ur Rehman Shanshan Tu Obaid ur Rehman Yongfeng Huang Chathura M. Sarathchandra Magurawalage Chin-Chen Chang |
spellingShingle |
Sadaqat ur Rehman Shanshan Tu Obaid ur Rehman Yongfeng Huang Chathura M. Sarathchandra Magurawalage Chin-Chen Chang Optimization of CNN through Novel Training Strategy for Visual Classification Problems Entropy CNN optimization image classification MRPROP training algorithm |
author_facet |
Sadaqat ur Rehman Shanshan Tu Obaid ur Rehman Yongfeng Huang Chathura M. Sarathchandra Magurawalage Chin-Chen Chang |
author_sort |
Sadaqat ur Rehman |
title |
Optimization of CNN through Novel Training Strategy for Visual Classification Problems |
title_short |
Optimization of CNN through Novel Training Strategy for Visual Classification Problems |
title_full |
Optimization of CNN through Novel Training Strategy for Visual Classification Problems |
title_fullStr |
Optimization of CNN through Novel Training Strategy for Visual Classification Problems |
title_full_unstemmed |
Optimization of CNN through Novel Training Strategy for Visual Classification Problems |
title_sort |
optimization of cnn through novel training strategy for visual classification problems |
publisher |
MDPI AG |
series |
Entropy |
issn |
1099-4300 |
publishDate |
2018-04-01 |
description |
The convolution neural network (CNN) has achieved state-of-the-art performance in many computer vision applications e.g., classification, recognition, detection, etc. However, the global optimization of CNN training is still a problem. Fast classification and training play a key role in the development of the CNN. We hypothesize that the smoother and optimized the training of a CNN goes, the more efficient the end result becomes. Therefore, in this paper, we implement a modified resilient backpropagation (MRPROP) algorithm to improve the convergence and efficiency of CNN training. Particularly, a tolerant band is introduced to avoid network overtraining, which is incorporated with the global best concept for weight updating criteria to allow the training algorithm of the CNN to optimize its weights more swiftly and precisely. For comparison, we present and analyze four different training algorithms for CNN along with MRPROP, i.e., resilient backpropagation (RPROP), Levenberg–Marquardt (LM), conjugate gradient (CG), and gradient descent with momentum (GDM). Experimental results showcase the merit of the proposed approach on a public face and skin dataset. |
topic |
CNN optimization image classification MRPROP training algorithm |
url |
http://www.mdpi.com/1099-4300/20/4/290 |
work_keys_str_mv |
AT sadaqaturrehman optimizationofcnnthroughnoveltrainingstrategyforvisualclassificationproblems AT shanshantu optimizationofcnnthroughnoveltrainingstrategyforvisualclassificationproblems AT obaidurrehman optimizationofcnnthroughnoveltrainingstrategyforvisualclassificationproblems AT yongfenghuang optimizationofcnnthroughnoveltrainingstrategyforvisualclassificationproblems AT chathuramsarathchandramagurawalage optimizationofcnnthroughnoveltrainingstrategyforvisualclassificationproblems AT chinchenchang optimizationofcnnthroughnoveltrainingstrategyforvisualclassificationproblems |
_version_ |
1725629717563834368 |