Gravitational-wave signals from binaries that contain spinning black holes in general include an asymmetry between the +m and -m multipoles that is not included in most signal models used in LIGO-Virgo-KAGRA analysis to date. This asymmetry manifests itself in out-of-plane recoil of the final black hole and its inclusion in signal models is necessary both to measure this recoil, but also to accurately measure the full spin information of each black hole. We present the first model of the antisymmetric contribution to the dominant coprecessing-frame signal multipole throughout inspiral, merger, and ringdown. We model the antisymmetric contribution in the frequency domain, and take advantage of the approximations that the antisymmetric amplitude can be modeled as a ratio of the (already modeled) symmetric amplitude, and analytic relationships between the symmetric and antisymmetric phases during the inspiral and ringdown. The model is tuned to single-spin numerical-relativity simulations up to mass-ratio 8 and spin magnitudes of 0.8, and has been implemented in a recent phenomenological model for use in the fourth LIGO-Virgo-KAGRA observing run. However, the procedure described here can be easily applied to other time- or frequency-domain models.