Explainable Deep Reinforcement Learning for Production Control

Abstract

Due to the growing number of variants and smaller batch sizes manufacturing companies have to cope with increasing material flow complexity. Thus, increasing the difficulty for production planning and control (PPC) to create a feasible and economic production plan. Despite significant advances in PPC research, current PPC systems do not yet sufficiently meet the industry’s requirements (e.g., decision quality, reaction time, user trust). However, recent progress in the digitalization of production systems results in an increased amount of data being collected, thus enabling the use of data-intensive applications technologies, e.g., machine learning (ML). ML provides new possibilities for PPC to handle increasing complexity caused by rising numbers of product variants paired with smaller lot sizes. At the same time, ML can increase the decision quality and reduce the reaction time to disturbances in the production system, e.g., machine breakdowns. Partly, ML models, e.g., artificial neural networks (ANN), are perceived as black-box models, resulting in reduced user’s trust in the decision proposed by an ML-based PPC system. The approach presented in this publication aims at a more functional and user-friendly PPC system by leveraging multi-agent reinforcement-learning (MARL), an accomplished approach within the field of ML-based production control, and approaches for explaining decisions made by reinforcement learning (RL) algorithms. With the help of MARL, short reaction time and high decision quality can be realized. Subsequently, the developed MARL system is combined with methods from the field of explainable Artificial Intelligence (XAI) to increase the users’ trust. The use case results show that with the help of the developed system, rule-based controls, which are often used in industry, can be outperformed while providing explainable decisions.