Real-time dynamic integrated process planning and scheduling with reconfigurable manufacturing cells via multi-agent reinforcement learning

Amid the transformation driven by Industry 4.0 and 5.0, manufacturing is rapidly advancing toward greater intelligence and flexibility. Reconfigurable Matrix-structured Manufacturing Systems (RMMS) improve adaptability through dynamic structural and resource reconfiguration, while Integrated Process Planning and Scheduling (IPPS) jointly optimizes process routes and scheduling for optimal resource allocation and responsiveness. This study focuses on Dynamic IPPS with Reconfigurable Manufacturing Cells (DIPPS-RMC) in RMMS, and proposes a real-time scheduling approach based on multi-agent Proximal Policy Optimization (PPO) to reduce average tardiness and enhance system efficiency. A Mixed Integer Linear Programming model is established to address the complexity of process flows and dynamic scheduling, providing a solid theoretical foundation. The scheduling problem is further formulated as a Partially Observable Markov Decision Process to capture the uncertainty and partial observability of real manufacturing environments. To alleviate the credit assignment problem and enhance inter-agent coordination, a delayed reward-sharing mechanism is designed. A multi-agent PPO algorithm with centralized training and decentralized execution is introduced, leveraging parallel environment sampling to improve training efficiency and generalization. Extensive experiments on 270 cases across 27 scenarios show that the proposed method outperforms state-of-the-art multi-agent reinforcement learning algorithms in training speed, generalization, and scheduling performance. Its application to real-world cases further demonstrates effective handling of dynamic job arrivals and RMC breakdowns, validating its robustness and practical utility. These results confirm the method's effectiveness and applicability in dynamic, complex manufacturing environments, offering an innovative solution for real-time scheduling in RMMS.