Energy-Flexible Job-Shop Scheduling Using Deep Reinforcement Learning

Abstract

Considering its high energy demand, the manufacturing industry has grand potential for demand response studies to increase the use of clean energy while reducing its own electricity cost. Production scheduling, driven by smart demand response services, plays a major role in adjusting the manufacturing sector to the volatile energy market. As a state-of-the-art method for scheduling problems, reinforcement learning has not yet been applied to the job-shop scheduling problem with demand response objectives. To address this gap, we conceptualize and implement deep reinforcement learning as a single-agent approach, combining energy cost and makespan minimization objectives. We consider makespan as an ancillary objective in order not to entirely abandon the timely completion of production operations while assigning different weights to both objectives and analyzing the resulting trade-offs between them. Our main contribution is the integration of the energy cost-related objective. We present two innovative reward functions, which consider the dynamic energy prices to select a job for the machine or allow the machine idle. The reinforcement learning agent finds optimal schedules determined by cumulative energy costs for benchmark scheduling cases from the literature.