Quantum probes are atomic sized devices mapping information of their environment to quantum-mechanical states. By improving measurements and at the same time minimizing perturbation of the environment, they form a central asset for quantum technologies. We realize spin-based quantum probes by immersing individual Cs atoms into an ultracold Rb bath. Controlling inelastic spin-exchange processes between the probe and bath allows us to map motional and thermal information onto quantum-spin states. We show that the steady-state spin population is well suited for absolute thermometry, reducing temperature measurements to detection of quantum-spin distributions. Moreover, we find that the information gain per inelastic collision can be maximized by accessing the nonequilibrium spin dynamic. Keeping the motional degree of freedom thermalized, individual spin-exchange collisions yield information about the gas quantum by quantum. We find that the sensitivity of this nonequilibrium quantum probing effectively beats the steady-state Cramér-Rao limit by almost an order of magnitude, while reducing the perturbation of the bath to only three quanta of angular momentum. Our work paves the way for local probing of quantum systems at the Heisenberg limit, and moreover, for optimizing measurement strategies via control of nonequilibrium dynamics. © 2020 authors. Published by the American Physical Society.