From the numerical solution of the time-dependent Schrödinger equation, we obtain the times of ionization and return of the laser-driven electron in high-order harmonic generation by probing the dynamics with a second harmonic field polarized orthogonal to the fundamental field and observing the harmonic emission in dependence on the two-color delay. Our retrieval method using complex-time evolution gives ionization and return times in excellent agreement with the quantum-orbit model, while a retrieval based on real-time classical dynamics can introduce substantial errors. Because of the imaginary parts, the harmonic signal polarized along the probe field is nonzero for any two-color delay. The tunneling time can be retrieved under an assumption for the return time. © 2013 American Physical Society.