Towards a Flexible Approach to Transfer Machine Operation Know-How from Experts to Beginners with AI

Abstract

Training new users at a production machine is a time intensive and expensive task. To reduce the effort in this task we examine the possibilities of enhancing a production machine with a system that is able to learn from its users and teach inexperienced users this knowledge: Self-Learning and Self-Explanatory Machine SLEM. The learning process of SLEM relies on watching an experienced user working on a machine using camera-based human activity recognition which predicts the acitivities based on the estimated human skeleton in the video stream. SLEM must be able to work with little data to reduce the learning time as much as possible. Thus, this paper shows that training an activity recognition model solely on one experienced individual’s actions can lead to comparatively high activity recognition accuracy despite the low data variety. The results show that training on a single-person dataset can reach relatively high accuracy levels and is a suitable way of training the model in the industrial setting. For the teaching process, in which the system has to compare the actual activities with the target acitivities to give feedback, the activity recognition has to run in real-time. Different amounts of input data for the activity recognition model are examined and lead to a configuration with little accuracy loss and sufficient latency performance.