DIN: A Decentralized Inexact Newton Algorithm for Consensus Optimization
Ghalkha, Abdulmomen; Ben Issaid, Chaouki; Elgabli, Anis; Bennis, Mehdi (2023-10-23)
IEEE
23.10.2023
A. Ghalkha, C. Ben Issaid, A. Elgabli and M. Bennis, "DIN: A Decentralized Inexact Newton Algorithm for Consensus Optimization," ICC 2023 - IEEE International Conference on Communications, Rome, Italy, 2023, pp. 4391-4396, doi: 10.1109/ICC45041.2023.10278721.
https://rightsstatements.org/vocab/InC/1.0/
© 2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
https://rightsstatements.org/vocab/InC/1.0/
© 2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
https://rightsstatements.org/vocab/InC/1.0/
Julkaisun pysyvä osoite on
https://urn.fi/URN:NBN:fi:oulu-202312183892
https://urn.fi/URN:NBN:fi:oulu-202312183892
Tiivistelmä
Abstract
In this paper, we consider a decentralized consensus optimization problem defined over a network of inter-connected devices that collaboratively solve the problem using only local data and information exchange with their neighbours. Despite their fast convergence, Newton-type methods require sending Hessian information between devices, making them communication inefficient while violating the devices' privacy. By formulating the Newton direction learning problem as a sum of separable functions subjected to a consensus constraint, our proposed approach learns an inexact Newton direction alongside the global model using the proximal primal-dual (Prox-PDA) algorithm. Our algorithm, coined DIN, avoids sharing Hessian information between devices since each device shares a model-sized vector, concealing the first- and second-order information, reducing the network's burden and improving communication and energy efficiencies. Numerical simulations corroborate that DIN exhibits higher communication efficiency in terms of communication rounds while consuming less communication and computation energy compared to existing second-order decentralized baselines.
In this paper, we consider a decentralized consensus optimization problem defined over a network of inter-connected devices that collaboratively solve the problem using only local data and information exchange with their neighbours. Despite their fast convergence, Newton-type methods require sending Hessian information between devices, making them communication inefficient while violating the devices' privacy. By formulating the Newton direction learning problem as a sum of separable functions subjected to a consensus constraint, our proposed approach learns an inexact Newton direction alongside the global model using the proximal primal-dual (Prox-PDA) algorithm. Our algorithm, coined DIN, avoids sharing Hessian information between devices since each device shares a model-sized vector, concealing the first- and second-order information, reducing the network's burden and improving communication and energy efficiencies. Numerical simulations corroborate that DIN exhibits higher communication efficiency in terms of communication rounds while consuming less communication and computation energy compared to existing second-order decentralized baselines.
Kokoelmat
- Avoin saatavuus [38840]