Concentrated Differentially Private Federated Learning With Performance Analysis
Federated learning engages a set of edge devices to collaboratively train a common model without sharing their local data and has advantage in user privacy over traditional cloud-based learning approaches. However, recent model inversion attacks and membership inference attacks have demonstrated tha...
Main Authors: | , , |
---|---|
Format: | Article |
Language: | English |
Published: |
IEEE
2021-01-01
|
Series: | IEEE Open Journal of the Computer Society |
Subjects: | |
Online Access: | https://ieeexplore.ieee.org/document/9495177/ |
id |
doaj-3cb21cb97e114876b9b31f1dd9ee2754 |
---|---|
record_format |
Article |
spelling |
doaj-3cb21cb97e114876b9b31f1dd9ee27542021-08-23T23:01:32ZengIEEEIEEE Open Journal of the Computer Society2644-12682021-01-01227628910.1109/OJCS.2021.30991089495177Concentrated Differentially Private Federated Learning With Performance AnalysisRui Hu0Yuanxiong Guo1https://orcid.org/0000-0003-2241-125XYanmin Gong2https://orcid.org/0000-0002-1761-2834Department of Electrical and Computer Engineering, University of Texas at San Antonio, San Antonio, TX, USADepartment of Information Systems and Cyber Security, University of Texas at San Antonio, San Antonio, TX, USADepartment of Electrical and Computer Engineering, University of Texas at San Antonio, San Antonio, TX, USAFederated learning engages a set of edge devices to collaboratively train a common model without sharing their local data and has advantage in user privacy over traditional cloud-based learning approaches. However, recent model inversion attacks and membership inference attacks have demonstrated that shared model updates during the interactive training process could still leak sensitive user information. Thus, it is desirable to provide rigorous differential privacy (DP) guarantee in federated learning. The main challenge to providing DP is to maintain high utility of federated learning model with repeatedly introduced randomness of DP mechanisms, especially when the server is not fully trusted. In this paper, we investigate how to provide DP to the most widely adopted federated learning scheme, federated averaging. Our approach combines local gradient perturbation, secure aggregation, and zero-concentrated differential privacy (zCDP) for better utility and privacy protection without a trusted server. We jointly consider the performance impacts of randomnesses introduced by the DP mechanism, client sampling and data subsampling in our approach, and theoretically analyze the convergence rate and end-to-end DP guarantee with non-convex loss functions. We also demonstrate that our proposed method has good utility-privacy trade-off through extensive numerical experiments on the real-world dataset.https://ieeexplore.ieee.org/document/9495177/Federated learningsecurity and privacyconvergence analysiszero-concentrated differential privacy |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Rui Hu Yuanxiong Guo Yanmin Gong |
spellingShingle |
Rui Hu Yuanxiong Guo Yanmin Gong Concentrated Differentially Private Federated Learning With Performance Analysis IEEE Open Journal of the Computer Society Federated learning security and privacy convergence analysis zero-concentrated differential privacy |
author_facet |
Rui Hu Yuanxiong Guo Yanmin Gong |
author_sort |
Rui Hu |
title |
Concentrated Differentially Private Federated Learning With Performance Analysis |
title_short |
Concentrated Differentially Private Federated Learning With Performance Analysis |
title_full |
Concentrated Differentially Private Federated Learning With Performance Analysis |
title_fullStr |
Concentrated Differentially Private Federated Learning With Performance Analysis |
title_full_unstemmed |
Concentrated Differentially Private Federated Learning With Performance Analysis |
title_sort |
concentrated differentially private federated learning with performance analysis |
publisher |
IEEE |
series |
IEEE Open Journal of the Computer Society |
issn |
2644-1268 |
publishDate |
2021-01-01 |
description |
Federated learning engages a set of edge devices to collaboratively train a common model without sharing their local data and has advantage in user privacy over traditional cloud-based learning approaches. However, recent model inversion attacks and membership inference attacks have demonstrated that shared model updates during the interactive training process could still leak sensitive user information. Thus, it is desirable to provide rigorous differential privacy (DP) guarantee in federated learning. The main challenge to providing DP is to maintain high utility of federated learning model with repeatedly introduced randomness of DP mechanisms, especially when the server is not fully trusted. In this paper, we investigate how to provide DP to the most widely adopted federated learning scheme, federated averaging. Our approach combines local gradient perturbation, secure aggregation, and zero-concentrated differential privacy (zCDP) for better utility and privacy protection without a trusted server. We jointly consider the performance impacts of randomnesses introduced by the DP mechanism, client sampling and data subsampling in our approach, and theoretically analyze the convergence rate and end-to-end DP guarantee with non-convex loss functions. We also demonstrate that our proposed method has good utility-privacy trade-off through extensive numerical experiments on the real-world dataset. |
topic |
Federated learning security and privacy convergence analysis zero-concentrated differential privacy |
url |
https://ieeexplore.ieee.org/document/9495177/ |
work_keys_str_mv |
AT ruihu concentrateddifferentiallyprivatefederatedlearningwithperformanceanalysis AT yuanxiongguo concentrateddifferentiallyprivatefederatedlearningwithperformanceanalysis AT yanmingong concentrateddifferentiallyprivatefederatedlearningwithperformanceanalysis |
_version_ |
1721198030782201856 |