A deep reinforcement learning based approach for dynamic job shop scheduling considering variable processing time

The job shop scheduling problem is a common challenge in intelligent manufacturing. In a real workshop environment, parameters like processing time often change dynamically. Scheduling strategies must be adjusted flexibly and quickly to match the current state. However, traditional methods only find the optimal solution for a specific instance. When environment parameters change, recalculations are required, leading to high time costs. To address these problems, a scheduling method called S2S-AC, based on deep reinforcement learning, is proposed to efficiently solve the dynamic job shop scheduling problem with variable processing times. In the proposed method, JSSP is modeled as a sequential decision-making problem using a Markov Decision Process. A new state set model, including four static states and two dynamic states, is designed with operations as the action set. An end-to-end framework that combines the Pointer Network model with the A2C algorithm is used to construct the DRL network, which is trained with multiple samples. The trained network directly outputs the scheduling strategy for new instances without requiring retraining. In static experiments, the effectiveness of S2S-AC is verified by comparing its solution results with those of SPT, LPT, MTWR, and the genetic algorithm on benchmark instances. In dynamic experiments, S2SAC achieved the best solution results in all randomly generated test instances based on instance ft10, compared to the above methods, with relatively short solution times.