Learning Virtual Borders through Semantic Scene Understanding and Augmented Reality

Abstract

Virtual borders are an opportunity to allow users the interactive restriction of their mobile robots' workspaces, e.g. to avoid navigation errors or to exclude certain areas from working. Currently, works in this field have focused on human-robot interaction (HRI) methods to restrict the workspace. However, recent trends towards smart environments and the tremendous progress in semantic scene understanding give new opportunities to enhance the HRI-based methods. Therefore, we propose a novel learning and support system (LSS) to support users during teaching of virtual borders. Our LSS learns from user interactions employing methods from visual scene understanding and supports users through recommendations for interactions. The bidirectional interaction between the user and system is realized using augmented reality. A validation of the approach shows that the LSS robustly recognizes a limited set of typical areas for virtual borders based on previous user interactions (F1 - Score= 91.5%) while preserving the high accuracy of standard HRI-based methods with a median of Mdn= 84.6%. Moreover, this approach allows the reduction of the interaction time to a constant mean value of M = 2 seconds making it independent of the border length. This avoids a linear interaction time of standard HRI-based methods.