Creating see-through image using two RGB-D sensors for remote control robot

Abstract

This paper presents a system which generates images to see through obstacles. For remote operation of robots in dangerous situations such as disaster sites, visual information affect the operational efficiency to a great extent. Diminished reality, which has been proposed in the field of computer vision, is a technique which generates images as if users saw through obstacles. The proposed method uses RGB-D sensors (RGB-D sensors can get not only RGB information as ordinary cameras can get, but also information of distance between the sensor and each point in the image) attached in front of and on the arm of the robot and enables to see through obstacles, letting the sensor move along with the arm. Using distance information from RGB-D sensors, coordinate transform is performed to produce images from arbitrary viewpoints. Since the method is simplified as much as possible, it is suitable for more general situation and requires smaller calculation cost than previous works. The experiments on a manipulator robot show the ability to see through obstacles from arbitrary viewpoints in real time.