Deep Reinforcement Learning-Based Control Strategy for Underwater Manipulator Systems

This paper presents a deep reinforcement learning-based control strategy for underwater manipulator systems, addressing the challenges of environmental disturbances and actuator faults, which is of vital importance for enhancing the human-computer interaction experience. A dynamic model of a single-joint underwater robotic arm is established using Newton-Euler formalism, incorporating the effects of added mass and hydrodynamic drag. A TD3 (Twin Delayed Deep Deterministic Policy Gradient) algorithm is developed for adaptive tracking, integrating a carefully designed reward function that balances tracking precision, control smoothness, and energy efficiency. Simulation results under five typical conditions, including disturbances and actuator faults, demonstrate that the proposed TD3 controller achieves high-precision tracking with strong adaptability and fault tolerance.