In recent decades, toolkits for innovation have been increasingly used to integrate users into new product development processes. They promise to empower users in these processes by providing design freedom and iterative learning for the transformation of ideas into products. Despite these potential benefits, little is known about how these often digital toolkits compare to traditional design methods, and what role previous experience of users in new product development plays. To compare the effectiveness of toolkits for innovation with physical product modeling, we conducted a two-stage laboratory experiment in the form of an innovation challenge. One hundred non-expert and 46 expert users created new product designs, which were subsequently evaluated by an independent jury. Our results show that users with no experience in digital or physical design tasks develop more innovative product ideas when using a digital toolkit for innovation than when they use physical product modeling, while expert users seem to be able to use both methods with equal success. We, thereby, show that toolkits for innovation are a powerful way to translate users' solution information into a prototype. Moreover, our results indicate that the usage of a digital toolkit for innovation decreases the quality gap between the designs of non-expert and expert users in new product development.