A Deep Reinforcement Learning-Based Decentralized Hierarchical Motion Control Strategy for Multiple Amphibious Spherical Robot Systems With Tilting Thrusters

He Yin, Shuxiang Guo*, Ao Li, Liwei Shi, Meng Liu

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

9 Citations (Scopus)

Abstract

The variable operating conditions and hostile environments faced by underwater robots remain a challenge for motion control in unknown environments. In order to improve the capability of the amphibious spherical robot (ASR) in unknown environments, a decentralized hierarchical deep reinforcement learning (DRL) motion control method based on deep deterministic policy gradient (DDPG) for multiple ASRs system is proposed. In the low-level, a DDPG-based motion controller is trained under a compound rewarding to learn to set the configurations of the tilting angle and rotational speed of each thruster at a proper timescale. At the high-level, a planning controller consisting of different action networks is designed to generate a reasonable thrust target to guide the movement of the robot. Specifically, inspired by the artificial potential field (APF) method, the complex underwater motion can be decomposed into several simple virtual forces. Each action network is trained to learn to generate a virtual thrust target component for a specific action. By combining the outputs of several action networks, the distributed cooperative motion control for multirobot systems can then be easily achieved. Finally, the motion control strategy is applied to the physical multi-ASR system, and the experiment results have shown satisfactory performance.

Original languageEnglish
Pages (from-to)769-779
Number of pages11
JournalIEEE Sensors Journal
Volume24
Issue number1
DOIs
Publication statusPublished - 1 Jan 2024

Keywords

  • Amphibious spherical robot (ASR)
  • collision avoidance
  • deep reinforcement learning (DRL)
  • motion control
  • multirobot system
  • thrust-vectoring
  • tilting thruster

Fingerprint

Dive into the research topics of 'A Deep Reinforcement Learning-Based Decentralized Hierarchical Motion Control Strategy for Multiple Amphibious Spherical Robot Systems With Tilting Thrusters'. Together they form a unique fingerprint.

Cite this