In Pursuit of Many: A Review of Modern Multiple Object Tracking Systems

Abstract

Multiple Object Tracking (MOT) is a core capability in modern computer vision, essential to autonomous driving, surveillance, sports analytics, robotics, and biomedical imaging. Persistent identity assignment across frames remains challenging in real scenes because of occlusion, dense crowds, appearance ambiguity, scale variation, camera motion, and identity switching. In this survey we synthesize recent progress by organizing methods around the problems they target and the paradigms they adopt. We cover the historical progression from tracking-by-detection to hybrid and end-to-end designs, and we summarize major architectural directions including transformer-based trackers, generative/diffusion formulations, state-space predictors, Siamese and graph-based models, and the growing impact of foundation models for detection and representation. We review benchmark trends that motivate method design, documenting the shift from saturated pedestrian benchmarks to challenge-driven and domain-specific datasets and we analyze evaluation practice by comparing classic and newer motion- and safety-centric metrics. Finally, we connect algorithmic trends to practical deployment constraints and outline emerging directions, foundation-model integration, open-vocabulary and multimodal tracking, unified evaluation, and domain-adaptive methods, that we believe will shape MOT research and real-world adoption.