Multimodal Interaction-aware Motion Prediction for Autonomous Street\n Crossing

Abstract

For mobile robots navigating on sidewalks, it is essential to be able to\nsafely cross street intersections. Most existing approaches rely on the\nrecognition of the traffic light signal to make an informed crossing decision.\nAlthough these approaches have been crucial enablers for urban navigation, the\ncapabilities of robots employing such approaches are still limited to\nnavigating only on streets containing signalized intersections. In this paper,\nwe address this challenge and propose a multimodal convolutional neural network\nframework to predict the safety of a street intersection for crossing. Our\narchitecture consists of two subnetworks; an interaction-aware trajectory\nestimation stream IA-TCNN, that predicts the future states of all observed\ntraffic participants in the scene, and a traffic light recognition stream\nAtteNet. Our IA-TCNN utilizes dilated causal convolutions to model the behavior\nof the observable dynamic agents in the scene without explicitly assigning\npriorities to the interactions among them. While AtteNet utilizes\nSqueeze-Excitation blocks to learn a content-aware mechanism for selecting the\nrelevant features from the data, thereby improving the noise robustness.\nLearned representations from the traffic light recognition stream are fused\nwith the estimated trajectories from the motion prediction stream to learn the\ncrossing decision. Furthermore, we extend our previously introduced Freiburg\nStreet Crossing dataset with sequences captured at different types of\nintersections, demonstrating complex interactions among the traffic\nparticipants. Extensive experimental evaluations on public benchmark datasets\nand our proposed dataset demonstrate that our network achieves state-of-the-art\nperformance for each of the subtasks, as well as for the crossing safety\nprediction.\n