Integrated process planning and scheduling with reconfigurable manufacturing cells through an improved dueling double deep Q-network algorithm

The increasing demand for integrated, flexible, and scalable engineering systems is accelerating the deep application of artificial intelligence in manufacturing scheduling. Such advancements help improve responsiveness and integration in the face of dynamic market conditions. This research addresses the Integrated Process Planning and Scheduling with Reconfigurable Manufacturing Cells (IPPS-RMCs) in the Matrix-structured Manufacturing System (MMS). To address this challenge, we propose an improved Dueling Double Deep Q-Network (D3QN) algorithm to minimize makespan. A mixed-integer programming model is formulated, and the scheduling process is modeled as a Markov Decision Process, incorporating 13-dimensional state features, a 36-dimensional compound dispatching rule action space, and a customized reward mechanism. A soft ε-greedy strategy is employed to balance exploration and exploitation according to the problem scale dynamically. Experiments conducted across three scenarios (ranging from 6 to 50 jobs and 6 to 20 manufacturing cells) demonstrate that the proposed Deep Reinforcement Learning approach outperforms the Deep Q-Network algorithm and scheduling rules in terms of solution quality. This research provides an efficient and adaptive solution for manufacturing environments subject to rapidly shifting market demands.