Adaptive Progressive Transformer-Based Trajectory Prediction Under Fine-Grained Trajectory-Scene Interaction Constraint

Abstract

Trajectory prediction is crucial in understanding human behavior around intelligent agents, such as self-driving vehicles or social robots. Nevertheless, conventional approaches often fall short in effectively modeling the intricate trajectory-scene interaction. As they typically rely on simple feature concatenation methods which can introduce scene information irrelevant to motion trajectories, this leads to insufficient understanding of the scene context and a decrease in prediction performance. To tackle the issue, we propose an Adaptive Progressive Transformer-based Trajectory Predictor, APT-TP, which precisely forecasts future trajectories under the fine-grained trajectory-scene interaction constraint. The scene semantic maps and trajectory heatmaps are initially fused at a coarse-grained semantic level through a cross-attention mechanism. Afterward, inverse reinforcement learning is introduced to learn the trajectory-scene interaction from the fused results and generate possible future path plans through the Gumbel-Softmax sampling strategy. Finally, the generated path plans regarding the trajectory-scene interaction constraint are fused with the refined motion features at a fine-grained policy level through a novel APFormer. An adaptive motion token extractor is used to mitigate the redundancy in the refined motion features. APFormer fuses motion information and path plans progressively to generate future trajectories. APT-TP achieves promising performance on two benchmark datasets, Stanford Drone Dataset (SDD) and ETH/UCY, revealing its superiority. Moreover, qualitative evaluations demonstrate its effectiveness in exploring the trajectory-scene interaction, which is beneficial for ameliorated trajectory prediction performance.