Automated guided vehicle dispatching and routing integration via digital twin with deep reinforcement learning

Lixiang Zhang, Chen Yang, Yan Yan, Ze Cai, Yaoguang Hu*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

13 Citations (Scopus)

Abstract

The manufacturing industry has witnessed a significant shift towards high flexibility and adaptability, driven by personalized demands. However, automated guided vehicle (AGV) dispatching optimization is still challenging when considering AGV routing with the spatial-temporal and kinematics constraints in intelligent production logistics systems, limiting the evolving industry applications. Against this backdrop, this paper presents a digital twin (DT)-enhanced deep reinforcement learning-based optimization framework to integrate AGV dispatching and routing at both horizontal and vertical levels. First, the proposed framework leverages a digital twin model of the shop floor to provide a simulation environment that closely mimics the actual manufacturing process, enabling the AGV dispatching agent to be trained in a realistic setting, thus reducing the risk of finding unrealistic solutions under specific shop-floor settings and preventing time-consuming trial-and-error processes. Then, the AGV dispatching with the routing problem is modeled as a Markov Decision Process to optimize tardiness and energy consumption. An improved dueling double deep Q network algorithm with count-based exploration is developed to learn a better-dispatching policy by interacting with the high-fidelity DT model that integrates a static path planning agent using A* and a dynamic collision avoidance agent using a deep deterministic policy gradient to prevent the congestion and deadlock. Experimental results show that our method outperforms four state-of-the-art methods with shorter tardiness, lower energy consumption, and better stability. The proposed method provides significant potential to utilize the digital twin and reinforcement learning in the decision-making and optimization of manufacturing processes.

Original languageEnglish
Pages (from-to)492-503
Number of pages12
JournalJournal of Manufacturing Systems
Volume72
DOIs
Publication statusPublished - Feb 2024

Keywords

  • Automated guided vehicle
  • Digital twin
  • Dispatching
  • Reinforcement learning
  • Routing

Fingerprint

Dive into the research topics of 'Automated guided vehicle dispatching and routing integration via digital twin with deep reinforcement learning'. Together they form a unique fingerprint.

Cite this