We propose and investigate a hybrid optomechanical system consisting of a micro-mechanical oscillator coupled to the internal states of a distant ensemble of atoms. The interaction between the systems is mediated by a light field which allows the coupling of the two systems in a modular way over long distances. Coupling to internal degrees of freedom of atoms opens up the possibility to employ high-frequency mechanical resonators in the MHz to GHz regime, such as optomechanical crystal structures, and to benefit from the rich toolbox of quantum control over internal atomic states. Previous schemes involving atomic motional states are rather limited in both of these aspects. We derive a full quantum model for the effective coupling including the main sources of decoherence. As an application we show that sympathetic ground-state cooling and strong coupling between the two systems is possible.