Selected technical issues of deep neural networks for image classification purposes

• Published in: Bulletin of the Polish Academy of Sciences: Technical Sciences
• Main Authors: M. Grochowski, A. Kwasigroch, A. Mikołajczyk
• Format: Article
• Language: English
• Published: Polish Academy of Sciences 2019-04-01
• Subjects: deep neural network; deep learning; image classification; batch normalization; transfer learning; dropout
• Online Access: https://journals.pan.pl/Content/112085/PDF/21_363-376_00946_Bpast.No.67-2_06.02.20.pdf

In recent years, deep learning and especially deep neural networks (DNN) have obtained amazing performance on a variety of problems, in particular in classification or pattern recognition. Among many kinds of DNNs, the convolutional neural networks (CNN) are most commonly used. However, due to their complexity, there are many problems related but not limited to optimizing network parameters, avoiding overfitting and ensuring good generalization abilities. Therefore, a number of methods have been proposed by the researchers to deal with these problems. In this paper, we present the results of applying different, recently developed methods to improve deep neural network training and operating. We decided to focus on the most popular CNN structures, namely on VGG based neural networks: VGG16, VGG11 and proposed by us VGG8. The tests were conducted on a real and very important problem of skin cancer detection. A publicly available dataset of skin lesions was used as a benchmark. We analyzed the influence of applying: dropout, batch normalization, model ensembling, and transfer learning. Moreover, the influence of the type of activation function was checked. In order to increase the objectivity of the results, each of the tested models was trained 6 times and their results were averaged. In addition, in order to mitigate the impact of the selection of learning, test and validation sets, k-fold validation was applied.