RoverNet: Vision-Based Adaptive Human-to-Robot Object Handovers

Abstract

Enabling dynamic human-to-robot handovers is a challenging task, requiring a combination of human pose estimation, motion prediction and generation of a suitable receiving robot trajectory. In this paper, we present a method, capable of predicting human motion during a handover process by utilizing a state-of-the-art pose estimation framework, a single RGB-D camera and a recurrent neural network. Additionally, we propose a method for humanoid robot control that adapts the corresponding receiving trajectory in real time. We evaluate the network for handover position prediction and show that it can accurately predict the goal location of the human hand during a handover. We also implement an adaptive humanoid robot control system that can facilitate a dynamic handover procedure.