Our work addresses the problem of long-term visual people tracking in complex environments. Tracking a varying number of objects entails the problem of associating detected objects to tracked targets. To overcome the data association problem, we apply a Tracking-by-Detection strategy that uses Randomized Forests as a classifier together with a Kalman filter. Randomized Forests build a strong classifier for multi-class problems through aggregating simple decision trees. Due to their modular setup, Randomized Forests can be built incrementally, which makes them useful for unsupervised learning of object features in real-time. New training samples can be incorporated on the fly, while not drifting away from previously learnt features. To support further analysis of the automatically generated trajectories, we annotate them with quality metrics based on the association confidence. To build the metrics we analyse the confidence values that derive from the Randomized Forests and the similarity of detected and tracked objects. We evaluate the performance of the overall approach with respect to available reference data of people crossing the scene.