Reinforcement learning-based trajectory planning for dual underwater manipulators

Dual underwater robotic manipulators are widely used in deep-sea exploration and subsea intervention tasks, where cooperative manipulation and environmental interaction impose significant challenges on trajectory planning. This paper investigates a reinforcement learning-based trajectory planning approach for dual underwater manipulators under kinematic constraints, collision avoidance requirements, and underwater disturbances. Proximal policy optimization (PPO) is employed to learn a continuous trajectory planning policy directly through interaction with the environment, benefiting from its stable training and robustness in high-dimensional control problems. To account for underwater operational characteristics, joint velocity is explicitly incorporated into the reward design to suppress aggressive motions, and mild flow disturbances are introduced during training to enhance robustness against environmental variability. Simulation results demonstrate that the proposed method can generate smooth, feasible, and coordinated trajectories for dual underwater manipulators under complex dynamics and underwater uncertainties.