CLORP: Cross-Layer Opportunistic Routing Protocol for Underwater Sensor Networks Based on Multi-Agent Reinforcement Learning

With the development of the Internet of Underwater Things (IoUT), both academia and industry have significant emphasized underwater wireless sensor networks (UWSNs). To address the issues of slow convergence, high latency, and limited energy in existing intelligent routing protocols in UWSNs, a cross-layer opportunistic routing protocol (CLORP) for underwater sensor networks based on multi-agent reinforcement learning (MARL) is proposed in this paper. First, CLORP combines the decision-making capability of multi-agent reinforcement learning with the idea of opportunistic routing to sequentially select a set of neighbors with larger values as potential forwarding nodes, thereby increasing the packet transmission success rate. Second, in the design of the MARL reward function, two reward functions for successful and unsuccessful packet transmission are designed jointly with cross-layer information to improve the routing protocol’s performance. Finally, two algorithmic optimization strategies, adaptive learning rate and Q-value initialization based on location and number of neighbors, are proposed to facilitate the faster adaptation of agents to the dynamic changes of network topology and accelerate CLORP convergence. The experimental results demonstrate that CLORP can increase algorithm convergence speed by 13.2%, reduce network energy consumption by 25%, and decrease network latency by 31.2%.