Performance Optimization of Semi-supervised Distributed Learning from Non-IID Data

• Main Authors: Chieh-Sheng Wang, 王捷生
• Other Authors: 逄愛君
• Format: Others
• Language: en_US
• Published: 2019
• Online Access: http://ndltd.ncl.edu.tw/handle/at73fd

碩士 === 國立臺灣大學 === 資訊工程學研究所 === 107 === Modern mobile end devices, such as smartphones and dash cams, are collecting numerous data that can be useful for various IoT and AI appli- cations. However, as the concern of privacy rises, users are not willing to provide their data to the server for further data analysis, which involves deep learning model training. Therefore, Google proposes Federated Learning, of- fering distributed end devices a way to train a deep learning model without passing their private data to a centralized server. However, in the real world circumstances, the data on the end devices are non-independent and identi- cally distributed (non-IID) such that it may cause weight divergence during training and eventually result in a considerable decrease in the model perfor- mance. In this thesis, we propose an innovative Federated learning scheme, in which we design a new operation called Federated Swapping (FedSwap) to replace some Federated Averaging (FedAvg) operations based on a few shared data during federated training to alleviate the adverse impact of weight divergence. We implement our method on both image classification using CIFAR-10 benchmark data and object detection using the real world video data. Experiment results show that the accuracy of image classification is increased by 3.8%, and the object detection task can be improved by 1.1%.