Multiple Reconfigurable Intelligent Surfaces Aided V2X Offloading Networks: A Federated Reinforcement Learning-Based Approach

With the emergence of a large number of computation intensive and delay sensitive on-vehicle applications, vehicle edge computing (VEC), as an emerging computing paradigm, aims to enhance computing performance by offloading computation tasks to vehicles and edge servers. However, the mobility of vehicles and obstacles which hamper the propagation of electromagnetic signals may result in poor vehicle-to-everything (V2X) channel quality. In this paper, we consider improving the wireless propagation environment by deploying reconfigurable intelligent surface (RIS) devices and study a task offloading strategy in VEC networks assisted by RIS. Under the constraints of vehicle-to-vehicle link reliability and minimum transmission rate requirement, we jointly optimize the RIS reflection coefficient matrix, task vehicle offloading mode and beamforming vector to minimize the average offloading delay of the entire system. Considering the non-convexity of the optimization problem and the isomorphism of task vehicle agents, we propose a feasible joint optimization algorithm based on multi-agent proximal policy optimization with federated learning framework (F-MAPPO). Additionally, this paper further develops a low-complexity F-MAPPO algorithm (LF-MAPPO) to address the large multidimensional state space problem in F-MAPPO algorithm. Numerical simulation results show that compared with centralized and decentralized MAPPO algorithms, the proposed algorithm achieves performance boost in convergence. Moreover, the proposed LF-MAPPO algorithm reduces the average task processing latency by 44.05% and 48.91% compared to the schemes without RIS devices and without vehicle-to-vehicle (V2V) link, which indicate the crucial role of introducing RIS and V2V task offloading in improving the system performance.