Reinforcement Learning-based Active Disturbance Rejection Control for Nonlinear Systems with Disturbance

Xiangyu Kong; Yuanqing Xia

doi:10.1109/CFASTA57821.2023.10243334

Reinforcement Learning-based Active Disturbance Rejection Control for Nonlinear Systems with Disturbance

Xiangyu Kong, Yuanqing Xia^*

^*Corresponding author for this work

School of Automation

Beijing Institute of Technology

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

This paper proposes a reinforcement learning-based active disturbance rejection controller (RL-ADRC) for trajectory tracking control of partially unknown nonlinear systems with external disturbances. It is also a complementary combination of RL and ADRC. In this method, an actor-critic-based RL algorithm is employed to explore an optimal and adaptive control strategy. Unlike traditional ADRC, which requires manual tuning of controller parameters, RL-ADRC utilizes an actor-critic network to approximate the optimal control strategy. By adopting the disturbance compensation philosophy from ADRC and integrating it with RL, RL-ADRC exhibits improved robustness against tracking errors, and the learning process is more efficient in terms of time.

Original language	English
Title of host publication	Proceedings of the 2nd Conference on Fully Actuated System Theory and Applications, CFASTA 2023
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	799-804
Number of pages	6
ISBN (Electronic)	9798350332162
DOIs	https://doi.org/10.1109/CFASTA57821.2023.10243334
Publication status	Published - 2023
Event	2nd Conference on Fully Actuated System Theory and Applications, CFASTA 2023 - Qingdao, China Duration: 14 Jul 2023 → 16 Jul 2023

Publication series

Name	Proceedings of the 2nd Conference on Fully Actuated System Theory and Applications, CFASTA 2023

Conference

Conference	2nd Conference on Fully Actuated System Theory and Applications, CFASTA 2023
Country/Territory	China
City	Qingdao
Period	14/07/23 → 16/07/23

Keywords

Active Disturbance Rejection Controller
Reinforcement learning
Trajectory Tracking Control

Access to Document

10.1109/CFASTA57821.2023.10243334

Cite this

Kong, X., & Xia, Y. (2023). Reinforcement Learning-based Active Disturbance Rejection Control for Nonlinear Systems with Disturbance. In Proceedings of the 2nd Conference on Fully Actuated System Theory and Applications, CFASTA 2023 (pp. 799-804). (Proceedings of the 2nd Conference on Fully Actuated System Theory and Applications, CFASTA 2023). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/CFASTA57821.2023.10243334

Kong, Xiangyu ; Xia, Yuanqing. / Reinforcement Learning-based Active Disturbance Rejection Control for Nonlinear Systems with Disturbance. Proceedings of the 2nd Conference on Fully Actuated System Theory and Applications, CFASTA 2023. Institute of Electrical and Electronics Engineers Inc., 2023. pp. 799-804 (Proceedings of the 2nd Conference on Fully Actuated System Theory and Applications, CFASTA 2023).

@inproceedings{1377c5a37f454417b9f833c2a8adde66,

title = "Reinforcement Learning-based Active Disturbance Rejection Control for Nonlinear Systems with Disturbance",

abstract = "This paper proposes a reinforcement learning-based active disturbance rejection controller (RL-ADRC) for trajectory tracking control of partially unknown nonlinear systems with external disturbances. It is also a complementary combination of RL and ADRC. In this method, an actor-critic-based RL algorithm is employed to explore an optimal and adaptive control strategy. Unlike traditional ADRC, which requires manual tuning of controller parameters, RL-ADRC utilizes an actor-critic network to approximate the optimal control strategy. By adopting the disturbance compensation philosophy from ADRC and integrating it with RL, RL-ADRC exhibits improved robustness against tracking errors, and the learning process is more efficient in terms of time.",

keywords = "Active Disturbance Rejection Controller, Reinforcement learning, Trajectory Tracking Control",

author = "Xiangyu Kong and Yuanqing Xia",

note = "Publisher Copyright: {\textcopyright} 2023 IEEE.; 2nd Conference on Fully Actuated System Theory and Applications, CFASTA 2023 ; Conference date: 14-07-2023 Through 16-07-2023",

year = "2023",

doi = "10.1109/CFASTA57821.2023.10243334",

language = "English",

series = "Proceedings of the 2nd Conference on Fully Actuated System Theory and Applications, CFASTA 2023",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "799--804",

booktitle = "Proceedings of the 2nd Conference on Fully Actuated System Theory and Applications, CFASTA 2023",

address = "United States",

}

Kong, X & Xia, Y 2023, Reinforcement Learning-based Active Disturbance Rejection Control for Nonlinear Systems with Disturbance. in Proceedings of the 2nd Conference on Fully Actuated System Theory and Applications, CFASTA 2023. Proceedings of the 2nd Conference on Fully Actuated System Theory and Applications, CFASTA 2023, Institute of Electrical and Electronics Engineers Inc., pp. 799-804, 2nd Conference on Fully Actuated System Theory and Applications, CFASTA 2023, Qingdao, China, 14/07/23. https://doi.org/10.1109/CFASTA57821.2023.10243334

Reinforcement Learning-based Active Disturbance Rejection Control for Nonlinear Systems with Disturbance. / Kong, Xiangyu; Xia, Yuanqing.
Proceedings of the 2nd Conference on Fully Actuated System Theory and Applications, CFASTA 2023. Institute of Electrical and Electronics Engineers Inc., 2023. p. 799-804 (Proceedings of the 2nd Conference on Fully Actuated System Theory and Applications, CFASTA 2023).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Reinforcement Learning-based Active Disturbance Rejection Control for Nonlinear Systems with Disturbance

AU - Kong, Xiangyu

AU - Xia, Yuanqing

PY - 2023

Y1 - 2023

N2 - This paper proposes a reinforcement learning-based active disturbance rejection controller (RL-ADRC) for trajectory tracking control of partially unknown nonlinear systems with external disturbances. It is also a complementary combination of RL and ADRC. In this method, an actor-critic-based RL algorithm is employed to explore an optimal and adaptive control strategy. Unlike traditional ADRC, which requires manual tuning of controller parameters, RL-ADRC utilizes an actor-critic network to approximate the optimal control strategy. By adopting the disturbance compensation philosophy from ADRC and integrating it with RL, RL-ADRC exhibits improved robustness against tracking errors, and the learning process is more efficient in terms of time.

AB - This paper proposes a reinforcement learning-based active disturbance rejection controller (RL-ADRC) for trajectory tracking control of partially unknown nonlinear systems with external disturbances. It is also a complementary combination of RL and ADRC. In this method, an actor-critic-based RL algorithm is employed to explore an optimal and adaptive control strategy. Unlike traditional ADRC, which requires manual tuning of controller parameters, RL-ADRC utilizes an actor-critic network to approximate the optimal control strategy. By adopting the disturbance compensation philosophy from ADRC and integrating it with RL, RL-ADRC exhibits improved robustness against tracking errors, and the learning process is more efficient in terms of time.

KW - Active Disturbance Rejection Controller

KW - Reinforcement learning

KW - Trajectory Tracking Control

UR - http://www.scopus.com/inward/record.url?scp=85173614924&partnerID=8YFLogxK

U2 - 10.1109/CFASTA57821.2023.10243334

DO - 10.1109/CFASTA57821.2023.10243334

M3 - Conference contribution

AN - SCOPUS:85173614924

T3 - Proceedings of the 2nd Conference on Fully Actuated System Theory and Applications, CFASTA 2023

SP - 799

EP - 804

BT - Proceedings of the 2nd Conference on Fully Actuated System Theory and Applications, CFASTA 2023

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2nd Conference on Fully Actuated System Theory and Applications, CFASTA 2023

Y2 - 14 July 2023 through 16 July 2023

ER -

Kong X, Xia Y. Reinforcement Learning-based Active Disturbance Rejection Control for Nonlinear Systems with Disturbance. In Proceedings of the 2nd Conference on Fully Actuated System Theory and Applications, CFASTA 2023. Institute of Electrical and Electronics Engineers Inc. 2023. p. 799-804. (Proceedings of the 2nd Conference on Fully Actuated System Theory and Applications, CFASTA 2023). doi: 10.1109/CFASTA57821.2023.10243334

Reinforcement Learning-based Active Disturbance Rejection Control for Nonlinear Systems with Disturbance

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this