Interactive Robots Control Using Mixed Reality

Abstract

The paper presents a mixed reality-based approach for interactive control of robotic manipulators and mobile platforms. In particular, we designed an interactive and understandable interface for human-robot interaction. The interface provides tools for robot path programming and visualizes it. Path visualization helps workers understand robot behavior, it is important for safety human-robot interaction. The paper presents an architecture of that system and the implementation for an industrial robot KUKA iiwa and mobile robot platform Plato. The main issue of a multi-platform system is related to the synchronization of coordinate frames for all elements. To deal with this problem we realized 3 setting options: manually, by a camera with markers, point clouds processing. We implemented our interface on Microsoft HoloLens and evaluated it on users.