GPU Parallel Method for Deep Learning Image Classification

• Published in: Jisuanji gongcheng
• Main Author: HAN Yanling, SHEN Siyang, XU Lijun, WANG Jing, ZHANG Yun, ZHOU Ruyan
• Format: Article
• Language: English
• Published: Editorial Office of Computer Engineering 2023-01-01
• Subjects: gpu parallel|ring all reduce algorithm|data parallelism|model parallelism|deep learning|image classification
• Online Access: https://www.ecice06.com/fileup/1000-3428/PDF/20230122.pdf

This paper proposes an improved ring all-reduce algorithm with lower time complexity to improve the low efficiency of multiple Graphics Processing Unit(GPU) parallel transmission in deep-learning image classification scenes.This algorithm optimizes the data transmission process based on interval pairing among nodes to alleviate the bandwidth loss of the traditional parameter server parallel structure.Second, as data parallelism makes it challenging to support large-scale network parameters and accelerate the delay, this paper utilizes the characteristics that the weight parameters of the deep learning backbone network are lower than those of the full connection layer, the synchronization overhead is small, the weight of the full connection layer is large, and the gradient transmission overhead is very high. The backbone network data is parallel, and the model of the full connection layer is parallel.The parallel data communication between nodes is achieved using the improved Ring All Reduce algorithm, which is used for image classification training.Experiments were conducted on two widely used datasets:Cifar10 and mini ImageNet.The experimental results show that the proposed algorithm can achieve a better acceleration effect while maintaining classification accuracy.Compared with data parallelism, the improved effect can almost reach 45%.