Deep Reinforcement Learning for Dynamic Flexible Job-Shop Scheduling with Automated Guided Vehicles

Zhenyu Hou; Lixiang Zhang; Yiheng Wang; Yaoguang Hu

doi:10.1007/978-981-97-0194-0_11

Deep Reinforcement Learning for Dynamic Flexible Job-Shop Scheduling with Automated Guided Vehicles

Zhenyu Hou, Lixiang Zhang, Yiheng Wang, Yaoguang Hu^*

^*Corresponding author for this work

School of Mechanical Engineering

Beijing Institute of Technology

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

The emergence of automated guided vehicles (AGV) has dramatically facilitated shop floor transportation, making production scheduling challenging. Meanwhile, the arrival of dynamic tasks increases the complexity of production scheduling. Therefore, this study focuses on the dynamic scheduling problem with AGVs under new order arrival. First, the mathematical model of dynamic flexible job-shop scheduling problems with AGVs (DFJSPA) is developed. Then, the DFJSPA is modeled as a Markov Decision Process (MDP), where processing and transportation tasks are considered. Next, we propose a dueling double deep Q network (D3QN) algorithm to optimize the problem. The evaluation results under nine scenarios demonstrate that the D3QN algorithm has less tardiness than the composite scheduling rules, which indicates that the D3QN algorithm can achieve high-efficiency decision-making in dynamic manufacturing systems.

Original language	English
Title of host publication	Proceedings of Industrial Engineering and Management - International Conference on Smart Manufacturing, Industrial and Logistics Engineering and Asian Conference of Management Science and Applications
Editors	Chen-Fu Chien, Runliang Dou, Li Luo
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	89-99
Number of pages	11
ISBN (Print)	9789819701933
DOIs	https://doi.org/10.1007/978-981-97-0194-0_11
Publication status	Published - 2024
Event	3rd International Conference on Smart Manufacturing, Industrial and Logistics Engineering, SMILE 2023 and the 7th Asian Conference of Management Science and Applications, ACMSA 2023 - Chengdu, China Duration: 17 Nov 2023 → 19 Nov 2023

Publication series

Name	Lecture Notes in Mechanical Engineering
ISSN (Print)	2195-4356
ISSN (Electronic)	2195-4364

Conference

Conference	3rd International Conference on Smart Manufacturing, Industrial and Logistics Engineering, SMILE 2023 and the 7th Asian Conference of Management Science and Applications, ACMSA 2023
Country/Territory	China
City	Chengdu
Period	17/11/23 → 19/11/23

Keywords

Dynamic scheduling
Flexible job-shop
Reinforcement learning

Access to Document

10.1007/978-981-97-0194-0_11

Cite this

Hou, Z., Zhang, L., Wang, Y., & Hu, Y. (2024). Deep Reinforcement Learning for Dynamic Flexible Job-Shop Scheduling with Automated Guided Vehicles. In C.-F. Chien, R. Dou, & L. Luo (Eds.), Proceedings of Industrial Engineering and Management - International Conference on Smart Manufacturing, Industrial and Logistics Engineering and Asian Conference of Management Science and Applications (pp. 89-99). (Lecture Notes in Mechanical Engineering). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-981-97-0194-0_11

Hou, Zhenyu ; Zhang, Lixiang ; Wang, Yiheng et al. / Deep Reinforcement Learning for Dynamic Flexible Job-Shop Scheduling with Automated Guided Vehicles. Proceedings of Industrial Engineering and Management - International Conference on Smart Manufacturing, Industrial and Logistics Engineering and Asian Conference of Management Science and Applications. editor / Chen-Fu Chien ; Runliang Dou ; Li Luo. Springer Science and Business Media Deutschland GmbH, 2024. pp. 89-99 (Lecture Notes in Mechanical Engineering).

@inproceedings{15b79bb4d1424d0ebe8e081887bf355b,

title = "Deep Reinforcement Learning for Dynamic Flexible Job-Shop Scheduling with Automated Guided Vehicles",

abstract = "The emergence of automated guided vehicles (AGV) has dramatically facilitated shop floor transportation, making production scheduling challenging. Meanwhile, the arrival of dynamic tasks increases the complexity of production scheduling. Therefore, this study focuses on the dynamic scheduling problem with AGVs under new order arrival. First, the mathematical model of dynamic flexible job-shop scheduling problems with AGVs (DFJSPA) is developed. Then, the DFJSPA is modeled as a Markov Decision Process (MDP), where processing and transportation tasks are considered. Next, we propose a dueling double deep Q network (D3QN) algorithm to optimize the problem. The evaluation results under nine scenarios demonstrate that the D3QN algorithm has less tardiness than the composite scheduling rules, which indicates that the D3QN algorithm can achieve high-efficiency decision-making in dynamic manufacturing systems.",

keywords = "Dynamic scheduling, Flexible job-shop, Reinforcement learning",

author = "Zhenyu Hou and Lixiang Zhang and Yiheng Wang and Yaoguang Hu",

note = "Publisher Copyright: {\textcopyright} The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024.; 3rd International Conference on Smart Manufacturing, Industrial and Logistics Engineering, SMILE 2023 and the 7th Asian Conference of Management Science and Applications, ACMSA 2023 ; Conference date: 17-11-2023 Through 19-11-2023",

year = "2024",

doi = "10.1007/978-981-97-0194-0_11",

language = "English",

isbn = "9789819701933",

series = "Lecture Notes in Mechanical Engineering",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "89--99",

editor = "Chen-Fu Chien and Runliang Dou and Li Luo",

booktitle = "Proceedings of Industrial Engineering and Management - International Conference on Smart Manufacturing, Industrial and Logistics Engineering and Asian Conference of Management Science and Applications",

address = "Germany",

}

Hou, Z, Zhang, L, Wang, Y & Hu, Y 2024, Deep Reinforcement Learning for Dynamic Flexible Job-Shop Scheduling with Automated Guided Vehicles. in C-F Chien, R Dou & L Luo (eds), Proceedings of Industrial Engineering and Management - International Conference on Smart Manufacturing, Industrial and Logistics Engineering and Asian Conference of Management Science and Applications. Lecture Notes in Mechanical Engineering, Springer Science and Business Media Deutschland GmbH, pp. 89-99, 3rd International Conference on Smart Manufacturing, Industrial and Logistics Engineering, SMILE 2023 and the 7th Asian Conference of Management Science and Applications, ACMSA 2023, Chengdu, China, 17/11/23. https://doi.org/10.1007/978-981-97-0194-0_11

Deep Reinforcement Learning for Dynamic Flexible Job-Shop Scheduling with Automated Guided Vehicles. / Hou, Zhenyu; Zhang, Lixiang; Wang, Yiheng et al.
Proceedings of Industrial Engineering and Management - International Conference on Smart Manufacturing, Industrial and Logistics Engineering and Asian Conference of Management Science and Applications. ed. / Chen-Fu Chien; Runliang Dou; Li Luo. Springer Science and Business Media Deutschland GmbH, 2024. p. 89-99 (Lecture Notes in Mechanical Engineering).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Deep Reinforcement Learning for Dynamic Flexible Job-Shop Scheduling with Automated Guided Vehicles

AU - Hou, Zhenyu

AU - Zhang, Lixiang

AU - Wang, Yiheng

AU - Hu, Yaoguang

N1 - Publisher Copyright: © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024.

PY - 2024

Y1 - 2024

N2 - The emergence of automated guided vehicles (AGV) has dramatically facilitated shop floor transportation, making production scheduling challenging. Meanwhile, the arrival of dynamic tasks increases the complexity of production scheduling. Therefore, this study focuses on the dynamic scheduling problem with AGVs under new order arrival. First, the mathematical model of dynamic flexible job-shop scheduling problems with AGVs (DFJSPA) is developed. Then, the DFJSPA is modeled as a Markov Decision Process (MDP), where processing and transportation tasks are considered. Next, we propose a dueling double deep Q network (D3QN) algorithm to optimize the problem. The evaluation results under nine scenarios demonstrate that the D3QN algorithm has less tardiness than the composite scheduling rules, which indicates that the D3QN algorithm can achieve high-efficiency decision-making in dynamic manufacturing systems.

AB - The emergence of automated guided vehicles (AGV) has dramatically facilitated shop floor transportation, making production scheduling challenging. Meanwhile, the arrival of dynamic tasks increases the complexity of production scheduling. Therefore, this study focuses on the dynamic scheduling problem with AGVs under new order arrival. First, the mathematical model of dynamic flexible job-shop scheduling problems with AGVs (DFJSPA) is developed. Then, the DFJSPA is modeled as a Markov Decision Process (MDP), where processing and transportation tasks are considered. Next, we propose a dueling double deep Q network (D3QN) algorithm to optimize the problem. The evaluation results under nine scenarios demonstrate that the D3QN algorithm has less tardiness than the composite scheduling rules, which indicates that the D3QN algorithm can achieve high-efficiency decision-making in dynamic manufacturing systems.

KW - Dynamic scheduling

KW - Flexible job-shop

KW - Reinforcement learning

UR - http://www.scopus.com/inward/record.url?scp=85193245130&partnerID=8YFLogxK

U2 - 10.1007/978-981-97-0194-0_11

DO - 10.1007/978-981-97-0194-0_11

M3 - Conference contribution

AN - SCOPUS:85193245130

SN - 9789819701933

T3 - Lecture Notes in Mechanical Engineering

SP - 89

EP - 99

BT - Proceedings of Industrial Engineering and Management - International Conference on Smart Manufacturing, Industrial and Logistics Engineering and Asian Conference of Management Science and Applications

A2 - Chien, Chen-Fu

A2 - Dou, Runliang

A2 - Luo, Li

PB - Springer Science and Business Media Deutschland GmbH

T2 - 3rd International Conference on Smart Manufacturing, Industrial and Logistics Engineering, SMILE 2023 and the 7th Asian Conference of Management Science and Applications, ACMSA 2023

Y2 - 17 November 2023 through 19 November 2023

ER -

Hou Z, Zhang L, Wang Y, Hu Y. Deep Reinforcement Learning for Dynamic Flexible Job-Shop Scheduling with Automated Guided Vehicles. In Chien CF, Dou R, Luo L, editors, Proceedings of Industrial Engineering and Management - International Conference on Smart Manufacturing, Industrial and Logistics Engineering and Asian Conference of Management Science and Applications. Springer Science and Business Media Deutschland GmbH. 2024. p. 89-99. (Lecture Notes in Mechanical Engineering). doi: 10.1007/978-981-97-0194-0_11

Deep Reinforcement Learning for Dynamic Flexible Job-Shop Scheduling with Automated Guided Vehicles

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this