Amplitude-Varying Perturbation for Balancing Privacy and Utility in Federated Learning

Yuan, X; Ni, W; Ding, M; Wei, K; Li, J; Poor, HV

Amplitude-Varying Perturbation for Balancing Privacy and Utility in Federated Learning

Yuan, X

Ni, W

Ding, M Wei, K Li, J Poor, HV

Permalink

Publisher:: IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
Publication Type:: Journal Article
Citation:: IEEE Transactions on Information Forensics and Security, 2023, 18, pp. 1884-1897
Issue Date:: 2023-01-01

Closed Access

	Filename	Description	Size
	Amplitude-Varying_Perturbation_for_Balancing_Privacy_and_Utility_in_Federated_Learning.pdf	Published version	10.61 MB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Yuan, X https://orcid.org/0000-0002-9167-1613
dc.contributor.author	Ni, W https://orcid.org/0000-0002-4933-594X
dc.contributor.author	Ding, M
dc.contributor.author	Wei, K
dc.contributor.author	Li, J
dc.contributor.author	Poor, HV
dc.date.accessioned	2024-03-27T07:46:45Z
dc.date.available	2024-03-27T07:46:45Z
dc.date.issued	2023-01-01
dc.identifier.citation	IEEE Transactions on Information Forensics and Security, 2023, 18, pp. 1884-1897
dc.identifier.issn	1556-6013
dc.identifier.issn	1556-6021
dc.identifier.uri	http://hdl.handle.net/10453/177281
dc.description.abstract	While preserving the privacy of federated learning (FL), differential privacy (DP) inevitably degrades the utility (i.e., accuracy) of FL due to model perturbations caused by DP noise added to model updates. Existing studies have considered exclusively noise with persistent root-mean-square amplitude and overlooked an opportunity of adjusting the amplitudes to alleviate the adverse effects of the noise. This paper presents a new DP perturbation mechanism with a time-varying noise amplitude to protect the privacy of FL and retain the capability of adjusting the learning performance. Specifically, we propose a geometric series form for the noise amplitude and reveal analytically the dependence of the series on the number of global aggregations and the $(\epsilon,\delta)$ -DP requirement. We derive an online refinement of the series to prevent FL from premature convergence resulting from excessive perturbation noise. Another important aspect is an upper bound developed for the loss function of a multi-layer perceptron (MLP) trained by FL running the new DP mechanism. Accordingly, the optimal number of global aggregations is obtained, balancing the learning and privacy. Extensive experiments are conducted using MLP, supporting vector machine, and convolutional neural network models on four public datasets. The contribution of the new DP mechanism to the convergence and accuracy of privacy-preserving FL is corroborated, compared to the state-of-the-art Gaussian noise mechanism with a persistent noise amplitude.
dc.language	English
dc.publisher	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
dc.relation.ispartof	IEEE Transactions on Information Forensics and Security
dc.relation.isbasedon	10.1109/TIFS.2023.3258255
dc.rights	info:eu-repo/semantics/closedAccess
dc.subject	08 Information and Computing Sciences, 09 Engineering
dc.subject.classification	Strategic, Defence & Security Studies
dc.subject.classification	40 Engineering
dc.subject.classification	46 Information and computing sciences
dc.title	Amplitude-Varying Perturbation for Balancing Privacy and Utility in Federated Learning
dc.type	Journal Article
utslib.citation.volume	18
utslib.for	08 Information and Computing Sciences
utslib.for	09 Engineering
pubs.organisational-group	University of Technology Sydney
pubs.organisational-group	University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	University of Technology Sydney/Faculty of Engineering and Information Technology/School of Electrical and Data Engineering
utslib.copyright.status	closed_access	*
dc.date.updated	2024-03-27T07:46:39Z
pubs.publication-status	Published
pubs.volume	18

Abstract:

While preserving the privacy of federated learning (FL), differential privacy (DP) inevitably degrades the utility (i.e., accuracy) of FL due to model perturbations caused by DP noise added to model updates. Existing studies have considered exclusively noise with persistent root-mean-square amplitude and overlooked an opportunity of adjusting the amplitudes to alleviate the adverse effects of the noise. This paper presents a new DP perturbation mechanism with a time-varying noise amplitude to protect the privacy of FL and retain the capability of adjusting the learning performance. Specifically, we propose a geometric series form for the noise amplitude and reveal analytically the dependence of the series on the number of global aggregations and the $(\epsilon,\delta)$ -DP requirement. We derive an online refinement of the series to prevent FL from premature convergence resulting from excessive perturbation noise. Another important aspect is an upper bound developed for the loss function of a multi-layer perceptron (MLP) trained by FL running the new DP mechanism. Accordingly, the optimal number of global aggregations is obtained, balancing the learning and privacy. Extensive experiments are conducted using MLP, supporting vector machine, and convolutional neural network models on four public datasets. The contribution of the new DP mechanism to the convergence and accuracy of privacy-preserving FL is corroborated, compared to the state-of-the-art Gaussian noise mechanism with a persistent noise amplitude.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/177281