Graph Attention-based MADRL for Access Control and Resource Allocation in Wireless Networked Control Systems

Abstract

Abstract

Wireless networked control systems (WNCS) offer great potential for revolutionizing the industrial automation by enabling wireless coordination between sensors, decision centers, and actuators. However, inefficient access control and resource allocation in WNCS are two critical factors that limit closed-loop performance and control stability, especially when the spectral and energy resources are limited. In this paper, we first analyze the optimal scheduling condition for maintaining control stability of a WNCS and then formulate a long-term optimization problem that jointly optimizes the access policy of edge devices, and grant policy and resource allocation at the edge server. We employ Lyapunov optimization to decompose the long-term optimization problem into a sequence of independent sub-problems, and propose a heterogeneous attention graph based multi-agent deep reinforcement learning algorithm that jointly optimizes the access and resource allocation policy. By leveraging the attention mechanism to project the graph representations from heterogeneous agents into a unified space, our proposed algorithm facilitates coordination among heterogeneous agents, thereby enhancing the overall system performance. Simulation results demonstrate that our proposed framework outperforms several benchmarks, validating its effectiveness.