Robust Person Identification and Following in a Mobile Robot Based on Deep Learning and Optical Tracking

Abstract

There is an exciting synergy between deep learning and robotics, combining the perception skills a deep learning system can achieve with the wide variety of physical responses a robot can perform. This article describes an embedded system integrated into a mobile robot capable of identifying and following a specific person reliably based on a convolutional neural network pipeline. In addition, the design incorporates an optical tracking system for supporting the inferences of the neural networks, allowing the determination of the position of a person using an RGB depth camera. The system runs on an NVIDIA Jetson TX2 board, an embedded System-on-Module capable of performing computationally demanding tasks onboard and handling the complexity needed to run a solid tracking and following algorithm. A robotic mobile base with the Jetson TX2 board attached receives velocity orders to move the system toward the target. The proposed approach has been validated on a mobile robotic platform that successfully follows a determined person, relying on the robustness of the combination of deep learning with optical tracking for working efficiently in a real environment.