Multi-Agent Reinforcement Learning with Cooperative Mechanism for Dynamic Job Shop Scheduling Problem

With the vigorous growth of the manufacturing industry, the complex dynamic scheduling problem has become the focus of enterprise production and cannot be ignored. Therefore, this paper established a mathematical model of the dynamic job shop scheduling problem DJSP with uncertain processing time. The optimi-zation objective was to minimize the makespan. Firstly, a new abstract algebraic structure scheduling group is proposed for the first time to represent the schedul-ing relationships. Then, a multi-agent reinforcement learning MADRL method was proposed, including two agents: proximal policy optimization PPO and deep Q-network DQN . The DQN changes the network structure under the co-operative mechanism obtaining samples with higher priority in experience replay. In PPO, state features are characterized by the scheduling solutions of jobs and machines. Different dispatching rules are assigned to feasible machines as action space, with the reward function defined by idle time during scheduling. Finally, the proposed method was compared with other heuristic dispatching rules through static and dynamic experiments. In various scales of instances, the results demonstrated significant performances, further validating the generality and supe-riority of the method.