An application to simulate and control industrial robot in virtual reality environment integrated with IR stereo camera sensor

摘要

The main goal of this research is to test for potential ways to control KUKA KR10 industrial arm manipulator using Virtual Reality technology and check for the advantages of applying this control methods. The final version of this application aims to achieve this goal by establishing an interaction between the user and the manipulator inside a virtual environment developed using the game engine Unity3D and the HTC VIVE Pro headset for the virtual visualization. By applying this control method, the user does not have to operate on site and instead he can work remotely. In addition to the ability to use off-line programming of the manipulator. The application is designed to simplify the controlling ways by displaying a complete virtual environment where the tridimensional model of the robotic arm can be visualized and programmed according to the real manipulator’s parameters and specifications. All the movements and parameters in the virtual environment can be synchronized with the real robot in an on-line or off-line path planning depending on the application or the task. The system integrates a set of virtual reality controllers and Leapmotion sensor as options to allow the user to control and see the robot and its parameters in the virtual environment. As a result of this research, the manipulator moves on the planned trajectory in a smooth way after applying some filtering techniques without losing its accuracy.