We investigate a fundamental property of device-independent security in quantum cryptography by characterizing probability distributions which are necessarily independent of the measurement results of any eavesdropper. We show that probability distributions that are secure in this sense are exactly the extremal quantum probability distributions. This allows us to give a characterization of security in algebraic terms. We apply the method to common examples for two-party as well as multiparty setups and present a scheme for verifying security of probability distributions with two parties, two measurement settings, and two outcomes.