We examine to what extent perturbative α′-corrections can generate a small cosmological constant in warped string compactifications. Focusing on the heterotic string at lowest order in the string loop expansion, we show that, for a maximally symmetric spacetime, the α′-corrected 4D scalar potential has no effect on the cosmological constant. The only relevant terms are instead higher order products of 4D Riemann tensors, which, however, are found to vanish in the usual perturbative regime of the α′-expansion. The heterotic string therefore only allows for 4D Minkowski vacua to all orders in α′, unless one also introduces string loop and/or nonperturbative corrections or allows for curvatures or field strengths that are large in string units. In particular, we find that perturbative α′-effects cannot induce weakly curved AdS4 solutions.