Communication and Energy Efficient Decentralized Learning Over D2D Networks
Liu, Shengli; Yu, Guanding; Wen, Dingzhu; Chen, Xianfu; Bennis, Mehdi; Chen, Hongyang (2023-05-05)
IEEE
05.05.2023
S. Liu, G. Yu, D. Wen, X. Chen, M. Bennis and H. Chen, "Communication and Energy Efficient Decentralized Learning Over D2D Networks," in IEEE Transactions on Wireless Communications, vol. 22, no. 12, pp. 9549-9563, Dec. 2023, doi: 10.1109/TWC.2023.3271854
https://rightsstatements.org/vocab/InC/1.0/
© 2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists,or reuse of any copyrighted component of this work in other works.
https://rightsstatements.org/vocab/InC/1.0/
© 2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists,or reuse of any copyrighted component of this work in other works.
https://rightsstatements.org/vocab/InC/1.0/
Julkaisun pysyvä osoite on
https://urn.fi/URN:NBN:fi:oulu-202401231425
https://urn.fi/URN:NBN:fi:oulu-202401231425
Tiivistelmä
Abstract
Device-to-device (D2D)-assisted decentralized learning has been proposed for mobile devices to collaboratively train artificial intelligence networks without the centralized parameter server. However, a densely connected network will cause large learning latency and energy consumption due to the limited computation and communication resources. In addition, link selection and aggregation weight have a significant impact on the learning performance. To cope with these challenges, we propose a joint computing power adjustment, wireless resource allocation, link selection, and aggregation weight adaptation mechanism to improve both communication and energy efficiencies. Specifically, the learning performances including the convergence rate, per-iteration learning latency, and per-iteration energy consumption are first analyzed. Then, an optimization problem is formulated to minimize the total learning cost, which is defined as the weighted sum of total learning latency and energy consumption. Given a network topology, the computing power and wireless resource allocation are optimized by the alternating optimization algorithm. Moreover, the optimal aggregation weight is obtained by semidefinite programming. With respect to link selection, we propose a tabu search based meta-heuristic algorithm to approximately achieve feasible solutions with a low computational complexity. Finally, extensive experiments demonstrate that the proposed link selection algorithm can significantly reduce the learning cost under the given learning accuracy requirement.
Device-to-device (D2D)-assisted decentralized learning has been proposed for mobile devices to collaboratively train artificial intelligence networks without the centralized parameter server. However, a densely connected network will cause large learning latency and energy consumption due to the limited computation and communication resources. In addition, link selection and aggregation weight have a significant impact on the learning performance. To cope with these challenges, we propose a joint computing power adjustment, wireless resource allocation, link selection, and aggregation weight adaptation mechanism to improve both communication and energy efficiencies. Specifically, the learning performances including the convergence rate, per-iteration learning latency, and per-iteration energy consumption are first analyzed. Then, an optimization problem is formulated to minimize the total learning cost, which is defined as the weighted sum of total learning latency and energy consumption. Given a network topology, the computing power and wireless resource allocation are optimized by the alternating optimization algorithm. Moreover, the optimal aggregation weight is obtained by semidefinite programming. With respect to link selection, we propose a tabu search based meta-heuristic algorithm to approximately achieve feasible solutions with a low computational complexity. Finally, extensive experiments demonstrate that the proposed link selection algorithm can significantly reduce the learning cost under the given learning accuracy requirement.
Kokoelmat
- Avoin saatavuus [38840]