The increasing number of variants in product portfolios contributes to the challenge of efficient manufacturing on production lines due to the resulting small batch sizes and thus frequent product changes that lower the average overall plant effectiveness. Especially for companies that manufacture at high speed on production lines, such as in the Fast Moving Consumer Good (FMCG) industry, it is a central task of operational management to increase the performance of production lines. Due to the multitude of different adjustment levers at several interdependent machines, the identification of efficient actions and their combination into economic improvement trajectories is challenging. There is a variety of approaches to address this challenge, e.g. simulation-based heuristics. However, these approaches mostly focus on details instead of giving a holistic perspective of the possibilities to improve a production line or are limited in practical application. In other areas of application, reinforcement learning has shown remarkable success in recent years. The principle feasibility of using reinforcement learning in this application context has been demonstrated as well. However, it became apparent that the integration of expert knowledge throughout the improvement process is necessary. For this reason this paper transforms five modules defined from an engineering point of view into the mathematical scheme of a markov decision problem, a default framework for reinforcement learning. This provides the foundation for applying reinforcement learning in combination with expert knowledge from an engineering perspective.