A Reinforcement Learning Approach for Integrated Scheduling in Automated Container Terminals

Zhanluo Zhang; Zilong Zhuang; Wei Qin; Huaijin Fang; Shulin Lan; Chen Yang; Yu Tian

doi:10.1109/IEEM55944.2022.9989692

A Reinforcement Learning Approach for Integrated Scheduling in Automated Container Terminals

Zhanluo Zhang, Zilong Zhuang, Wei Qin^*, Huaijin Fang, Shulin Lan, Chen Yang, Yu Tian

^*Corresponding author for this work

School of Cyberspace Science and Technology

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

4 Citations (Scopus)

Abstract

Automated container terminals are complex systems with multiple interactions and high dynamic characteristics. Integrated scheduling is expected to improve the overall efficiency. However, traditional optimization approaches such as mathematical models and meta-heuristic algorithms failed to tackle high dynamics. A reinforcement learning approach based on the scheduling network method is presented in this paper. Network-based heuristic rules are introduced into the action space, and a novel state definition that integrates local and global information about the scheduling problem is proposed. Group training and group validating strategies are adopted to test the generalization ability. Numerical experiment results reveal that the proposed approach converges to a high level and maintains good performance on unseen instances. Compared to the selected heuristic rules, the proposed method achieves 2.37% and 6.06% better results on training and test instances, respectively.

Original language	English
Title of host publication	IEEE International Conference on Industrial Engineering and Engineering Management, IEEM 2022
Publisher	IEEE Computer Society
Pages	1182-1186
Number of pages	5
ISBN (Electronic)	9781665486873
DOIs	https://doi.org/10.1109/IEEM55944.2022.9989692
Publication status	Published - 2022
Event	2022 IEEE International Conference on Industrial Engineering and Engineering Management, IEEM 2022 - Kuala Lumpur, Malaysia Duration: 7 Dec 2022 → 10 Dec 2022

Publication series

Name	IEEE International Conference on Industrial Engineering and Engineering Management
Volume	2022-December
ISSN (Print)	2157-3611
ISSN (Electronic)	2157-362X

Conference

Conference	2022 IEEE International Conference on Industrial Engineering and Engineering Management, IEEM 2022
Country/Territory	Malaysia
City	Kuala Lumpur
Period	7/12/22 → 10/12/22

Keywords

Automated container terminal
integrated scheduling
network scheduling
reinforcement learning

Access to Document

10.1109/IEEM55944.2022.9989692

Cite this

Zhang, Z., Zhuang, Z., Qin, W., Fang, H., Lan, S., Yang, C., & Tian, Y. (2022). A Reinforcement Learning Approach for Integrated Scheduling in Automated Container Terminals. In IEEE International Conference on Industrial Engineering and Engineering Management, IEEM 2022 (pp. 1182-1186). (IEEE International Conference on Industrial Engineering and Engineering Management; Vol. 2022-December). IEEE Computer Society. https://doi.org/10.1109/IEEM55944.2022.9989692

@inproceedings{c0a0bd037cac4926a1d91009ddb5b9a0,

title = "A Reinforcement Learning Approach for Integrated Scheduling in Automated Container Terminals",

abstract = "Automated container terminals are complex systems with multiple interactions and high dynamic characteristics. Integrated scheduling is expected to improve the overall efficiency. However, traditional optimization approaches such as mathematical models and meta-heuristic algorithms failed to tackle high dynamics. A reinforcement learning approach based on the scheduling network method is presented in this paper. Network-based heuristic rules are introduced into the action space, and a novel state definition that integrates local and global information about the scheduling problem is proposed. Group training and group validating strategies are adopted to test the generalization ability. Numerical experiment results reveal that the proposed approach converges to a high level and maintains good performance on unseen instances. Compared to the selected heuristic rules, the proposed method achieves 2.37% and 6.06% better results on training and test instances, respectively.",

keywords = "Automated container terminal, integrated scheduling, network scheduling, reinforcement learning",

author = "Zhanluo Zhang and Zilong Zhuang and Wei Qin and Huaijin Fang and Shulin Lan and Chen Yang and Yu Tian",

note = "Publisher Copyright: {\textcopyright} 2022 IEEE.; 2022 IEEE International Conference on Industrial Engineering and Engineering Management, IEEM 2022 ; Conference date: 07-12-2022 Through 10-12-2022",

year = "2022",

doi = "10.1109/IEEM55944.2022.9989692",

language = "English",

series = "IEEE International Conference on Industrial Engineering and Engineering Management",

publisher = "IEEE Computer Society",

pages = "1182--1186",

booktitle = "IEEE International Conference on Industrial Engineering and Engineering Management, IEEM 2022",

address = "United States",

}

Zhang, Z, Zhuang, Z, Qin, W, Fang, H, Lan, S, Yang, C & Tian, Y 2022, A Reinforcement Learning Approach for Integrated Scheduling in Automated Container Terminals. in IEEE International Conference on Industrial Engineering and Engineering Management, IEEM 2022. IEEE International Conference on Industrial Engineering and Engineering Management, vol. 2022-December, IEEE Computer Society, pp. 1182-1186, 2022 IEEE International Conference on Industrial Engineering and Engineering Management, IEEM 2022, Kuala Lumpur, Malaysia, 7/12/22. https://doi.org/10.1109/IEEM55944.2022.9989692

A Reinforcement Learning Approach for Integrated Scheduling in Automated Container Terminals. / Zhang, Zhanluo; Zhuang, Zilong; Qin, Wei et al.
IEEE International Conference on Industrial Engineering and Engineering Management, IEEM 2022. IEEE Computer Society, 2022. p. 1182-1186 (IEEE International Conference on Industrial Engineering and Engineering Management; Vol. 2022-December).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - A Reinforcement Learning Approach for Integrated Scheduling in Automated Container Terminals

AU - Zhang, Zhanluo

AU - Zhuang, Zilong

AU - Qin, Wei

AU - Fang, Huaijin

AU - Lan, Shulin

AU - Yang, Chen

AU - Tian, Yu

PY - 2022

Y1 - 2022

N2 - Automated container terminals are complex systems with multiple interactions and high dynamic characteristics. Integrated scheduling is expected to improve the overall efficiency. However, traditional optimization approaches such as mathematical models and meta-heuristic algorithms failed to tackle high dynamics. A reinforcement learning approach based on the scheduling network method is presented in this paper. Network-based heuristic rules are introduced into the action space, and a novel state definition that integrates local and global information about the scheduling problem is proposed. Group training and group validating strategies are adopted to test the generalization ability. Numerical experiment results reveal that the proposed approach converges to a high level and maintains good performance on unseen instances. Compared to the selected heuristic rules, the proposed method achieves 2.37% and 6.06% better results on training and test instances, respectively.

AB - Automated container terminals are complex systems with multiple interactions and high dynamic characteristics. Integrated scheduling is expected to improve the overall efficiency. However, traditional optimization approaches such as mathematical models and meta-heuristic algorithms failed to tackle high dynamics. A reinforcement learning approach based on the scheduling network method is presented in this paper. Network-based heuristic rules are introduced into the action space, and a novel state definition that integrates local and global information about the scheduling problem is proposed. Group training and group validating strategies are adopted to test the generalization ability. Numerical experiment results reveal that the proposed approach converges to a high level and maintains good performance on unseen instances. Compared to the selected heuristic rules, the proposed method achieves 2.37% and 6.06% better results on training and test instances, respectively.

KW - Automated container terminal

KW - integrated scheduling

KW - network scheduling

KW - reinforcement learning

UR - http://www.scopus.com/inward/record.url?scp=85146339145&partnerID=8YFLogxK

U2 - 10.1109/IEEM55944.2022.9989692

DO - 10.1109/IEEM55944.2022.9989692

M3 - Conference contribution

AN - SCOPUS:85146339145

T3 - IEEE International Conference on Industrial Engineering and Engineering Management

SP - 1182

EP - 1186

BT - IEEE International Conference on Industrial Engineering and Engineering Management, IEEM 2022

PB - IEEE Computer Society

T2 - 2022 IEEE International Conference on Industrial Engineering and Engineering Management, IEEM 2022

Y2 - 7 December 2022 through 10 December 2022

ER -

Zhang Z, Zhuang Z, Qin W, Fang H, Lan S, Yang C et al. A Reinforcement Learning Approach for Integrated Scheduling in Automated Container Terminals. In IEEE International Conference on Industrial Engineering and Engineering Management, IEEM 2022. IEEE Computer Society. 2022. p. 1182-1186. (IEEE International Conference on Industrial Engineering and Engineering Management). doi: 10.1109/IEEM55944.2022.9989692

A Reinforcement Learning Approach for Integrated Scheduling in Automated Container Terminals

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this