Local stochastic ADMM for communication-efficient distributed learning

Ben Issaid, Chaouki; Elgabli, Anis; Bennis, Mehdi (2022-05-16)
 
Avaa tiedosto
Lataukset: 

URL:
https://doi.org/10.1109/wcnc51071.2022.9771559

Ben Issaid, Chaouki
Elgabli, Anis
Bennis, Mehdi
Institute of Electrical and Electronics Engineers
16.05.2022

C. ben Issaid, A. Elgabli and M. Bennis, "Local Stochastic ADMM for Communication-Efficient Distributed Learning," 2022 IEEE Wireless Communications and Networking Conference (WCNC), Austin, TX, USA, 2022, pp. 1880-1885, doi: 10.1109/WCNC51071.2022.9771559

https://rightsstatements.org/vocab/InC/1.0/
© 2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
https://rightsstatements.org/vocab/InC/1.0/
doi:https://doi.org/10.1109/wcnc51071.2022.9771559
Näytä kaikki kuvailutiedot
Julkaisun pysyvä osoite on
https://urn.fi/URN:NBN:fi-fe2023020926609
Tiivistelmä

Abstract

In this paper, we propose a communication-efficient alternating direction method of multipliers (ADMM)-based algorithm for solving a distributed learning problem in the stochastic non-convex setting. Our approach runs a few stochastic gradient descent (SGD) steps to solve the local problem at each worker instead of finding the exact/approximate solution as proposed by existing ADMM-based works. By doing so, the proposed framework strikes a good balance between the computation and communication costs. Extensive simulation results show that our algorithm significantly outperforms existing stochastic ADMM in terms of communication-efficiency, notably in the presence of non-independent and identically distributed (non-IID) data.

Kokoelmat