Dynamic flexible job shop scheduling through multi-agent reinforcement learning: considering opportunistic maintenance and transportation time

Reinforcement learning has emerged as a crucial approach for addressing flexible job shop scheduling problems (FJSP), enabling real-time decision-making. However, current research mainly focuses on two-stage methods for FJSP considering machine maintenance in static environments. This leaves a research gap in integrating FJSP with opportunistic maintenance and transportation time under dynamic conditions. This oversight reduces machine availability and limits the applicability of scheduling solutions in practice. Therefore, this paper proposes a multi-agent reinforcement learning-based scheduling approach for FJSP in dynamic environments. The approach coordinates distributed job and machine agents to handle opportunistic maintenance and transportation constraints. The FJSP with maintenance and transportation constraints is modeled as a series of partially observable Markov Decision Processes for machine allocation and task sequencing. This model includes observation design, formulation of actions, design of reward functions for processing and maintenance tasks, and implementation of a decision-making policy using dueling deep neural networks. Then, an advanced dueling double deep Q-network algorithm with highlight experience replay technique is developed to accelerate learning and enhance stability, thereby reducing machine failures. The effectiveness of this approach is demonstrated through comparisons with traditional methods using condition-based maintenance strategies, demonstrating improved performance in numerical experiments and a real-world case study. This research highlights the potential of multi-agent reinforcement learning for real-time decision-making in multi-task scheduling.