An Obstacle Avoidance Method Using Asynchronous Policy-based Deep Reinforcement Learning with Discrete Action

Yuechuan Wang; Fenxi Yao; Lingguo Cui; Senchun Chai

doi:10.1109/CCDC55256.2022.10033562

An Obstacle Avoidance Method Using Asynchronous Policy-based Deep Reinforcement Learning with Discrete Action

Yuechuan Wang^*, Fenxi Yao^*, Lingguo Cui, Senchun Chai

^*Corresponding author for this work

School of Automation

Beijing Institute of Technology

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

4 Citations (Scopus)

Abstract

With the increasing application of mobile robots in manufacturing, service, and military fields, the demand of intelligent autonomous decision is also growing. In this paper, the state-of-the-art policy-based deep reinforcement learning (DRL) algorithm is applied to the mobile robot obstacle avoidance task. To solve the strong coupling in the existing DRL-based training process of mobile robot decision, an asynchronous decoupling architecture is proposed in this paper, which greatly improves the scalability and sample generation efficiency of the DRL algorithm. At the same time, based on the asynchronous decoupling architecture and Soft Actor-Critic (SAC) algorithm, we design an obstacle avoidance method named asynchronous dueling-based discrete action SAC (ADDSAC). Experiments show both action discretization and dueling network are simple and powerful techniques to improve obstacle avoidance performance. Finally, the reasons are also analyzed from different perspectives.

Original language	English
Title of host publication	Proceedings of the 34th Chinese Control and Decision Conference, CCDC 2022
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	6235-6241
Number of pages	7
ISBN (Electronic)	9781665478960
DOIs	https://doi.org/10.1109/CCDC55256.2022.10033562
Publication status	Published - 2022
Event	34th Chinese Control and Decision Conference, CCDC 2022 - Hefei, China Duration: 15 Aug 2022 → 17 Aug 2022

Publication series

Name	Proceedings of the 34th Chinese Control and Decision Conference, CCDC 2022

Conference

Conference	34th Chinese Control and Decision Conference, CCDC 2022
Country/Territory	China
City	Hefei
Period	15/08/22 → 17/08/22

Keywords

Mobile robot
Soft Actor-Critic
deep reinforcement learning
obstacle avoidance

Access to Document

10.1109/CCDC55256.2022.10033562

Cite this

Wang, Y., Yao, F., Cui, L., & Chai, S. (2022). An Obstacle Avoidance Method Using Asynchronous Policy-based Deep Reinforcement Learning with Discrete Action. In Proceedings of the 34th Chinese Control and Decision Conference, CCDC 2022 (pp. 6235-6241). (Proceedings of the 34th Chinese Control and Decision Conference, CCDC 2022). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/CCDC55256.2022.10033562

Wang, Yuechuan ; Yao, Fenxi ; Cui, Lingguo et al. / An Obstacle Avoidance Method Using Asynchronous Policy-based Deep Reinforcement Learning with Discrete Action. Proceedings of the 34th Chinese Control and Decision Conference, CCDC 2022. Institute of Electrical and Electronics Engineers Inc., 2022. pp. 6235-6241 (Proceedings of the 34th Chinese Control and Decision Conference, CCDC 2022).

@inproceedings{15726af49d344eaa8d161421efdca586,

title = "An Obstacle Avoidance Method Using Asynchronous Policy-based Deep Reinforcement Learning with Discrete Action",

abstract = "With the increasing application of mobile robots in manufacturing, service, and military fields, the demand of intelligent autonomous decision is also growing. In this paper, the state-of-the-art policy-based deep reinforcement learning (DRL) algorithm is applied to the mobile robot obstacle avoidance task. To solve the strong coupling in the existing DRL-based training process of mobile robot decision, an asynchronous decoupling architecture is proposed in this paper, which greatly improves the scalability and sample generation efficiency of the DRL algorithm. At the same time, based on the asynchronous decoupling architecture and Soft Actor-Critic (SAC) algorithm, we design an obstacle avoidance method named asynchronous dueling-based discrete action SAC (ADDSAC). Experiments show both action discretization and dueling network are simple and powerful techniques to improve obstacle avoidance performance. Finally, the reasons are also analyzed from different perspectives.",

keywords = "Mobile robot, Soft Actor-Critic, deep reinforcement learning, obstacle avoidance",

author = "Yuechuan Wang and Fenxi Yao and Lingguo Cui and Senchun Chai",

note = "Publisher Copyright: {\textcopyright} 2022 IEEE.; 34th Chinese Control and Decision Conference, CCDC 2022 ; Conference date: 15-08-2022 Through 17-08-2022",

year = "2022",

doi = "10.1109/CCDC55256.2022.10033562",

language = "English",

series = "Proceedings of the 34th Chinese Control and Decision Conference, CCDC 2022",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "6235--6241",

booktitle = "Proceedings of the 34th Chinese Control and Decision Conference, CCDC 2022",

address = "United States",

}

Wang, Y, Yao, F , Cui, L & Chai, S 2022, An Obstacle Avoidance Method Using Asynchronous Policy-based Deep Reinforcement Learning with Discrete Action. in Proceedings of the 34th Chinese Control and Decision Conference, CCDC 2022. Proceedings of the 34th Chinese Control and Decision Conference, CCDC 2022, Institute of Electrical and Electronics Engineers Inc., pp. 6235-6241, 34th Chinese Control and Decision Conference, CCDC 2022, Hefei, China, 15/08/22. https://doi.org/10.1109/CCDC55256.2022.10033562

An Obstacle Avoidance Method Using Asynchronous Policy-based Deep Reinforcement Learning with Discrete Action. / Wang, Yuechuan; Yao, Fenxi ; Cui, Lingguo et al.
Proceedings of the 34th Chinese Control and Decision Conference, CCDC 2022. Institute of Electrical and Electronics Engineers Inc., 2022. p. 6235-6241 (Proceedings of the 34th Chinese Control and Decision Conference, CCDC 2022).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - An Obstacle Avoidance Method Using Asynchronous Policy-based Deep Reinforcement Learning with Discrete Action

AU - Wang, Yuechuan

AU - Yao, Fenxi

AU - Cui, Lingguo

AU - Chai, Senchun

PY - 2022

Y1 - 2022

N2 - With the increasing application of mobile robots in manufacturing, service, and military fields, the demand of intelligent autonomous decision is also growing. In this paper, the state-of-the-art policy-based deep reinforcement learning (DRL) algorithm is applied to the mobile robot obstacle avoidance task. To solve the strong coupling in the existing DRL-based training process of mobile robot decision, an asynchronous decoupling architecture is proposed in this paper, which greatly improves the scalability and sample generation efficiency of the DRL algorithm. At the same time, based on the asynchronous decoupling architecture and Soft Actor-Critic (SAC) algorithm, we design an obstacle avoidance method named asynchronous dueling-based discrete action SAC (ADDSAC). Experiments show both action discretization and dueling network are simple and powerful techniques to improve obstacle avoidance performance. Finally, the reasons are also analyzed from different perspectives.

AB - With the increasing application of mobile robots in manufacturing, service, and military fields, the demand of intelligent autonomous decision is also growing. In this paper, the state-of-the-art policy-based deep reinforcement learning (DRL) algorithm is applied to the mobile robot obstacle avoidance task. To solve the strong coupling in the existing DRL-based training process of mobile robot decision, an asynchronous decoupling architecture is proposed in this paper, which greatly improves the scalability and sample generation efficiency of the DRL algorithm. At the same time, based on the asynchronous decoupling architecture and Soft Actor-Critic (SAC) algorithm, we design an obstacle avoidance method named asynchronous dueling-based discrete action SAC (ADDSAC). Experiments show both action discretization and dueling network are simple and powerful techniques to improve obstacle avoidance performance. Finally, the reasons are also analyzed from different perspectives.

KW - Mobile robot

KW - Soft Actor-Critic

KW - deep reinforcement learning

KW - obstacle avoidance

UR - http://www.scopus.com/inward/record.url?scp=85149552149&partnerID=8YFLogxK

U2 - 10.1109/CCDC55256.2022.10033562

DO - 10.1109/CCDC55256.2022.10033562

M3 - Conference contribution

AN - SCOPUS:85149552149

T3 - Proceedings of the 34th Chinese Control and Decision Conference, CCDC 2022

SP - 6235

EP - 6241

BT - Proceedings of the 34th Chinese Control and Decision Conference, CCDC 2022

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 34th Chinese Control and Decision Conference, CCDC 2022

Y2 - 15 August 2022 through 17 August 2022

ER -

Wang Y, Yao F , Cui L , Chai S. An Obstacle Avoidance Method Using Asynchronous Policy-based Deep Reinforcement Learning with Discrete Action. In Proceedings of the 34th Chinese Control and Decision Conference, CCDC 2022. Institute of Electrical and Electronics Engineers Inc. 2022. p. 6235-6241. (Proceedings of the 34th Chinese Control and Decision Conference, CCDC 2022). doi: 10.1109/CCDC55256.2022.10033562

An Obstacle Avoidance Method Using Asynchronous Policy-based Deep Reinforcement Learning with Discrete Action

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this