Occlusion-Free Visual Servoing for the Shared Autonomy Teleoperation of Dual-Arm Robots

Abstract

Visual servoing in telerobotics provides information to the operator about the remote location and assists in the task execution to reduce stress on the user. Occlusions in this scenario can, on the one hand, lead to the visual servoing failure, and, on the other hand, degrade the user experience and navigation performance due to the obstructed vision of elements of interest. In this letter, we consider a teleoperation system composed of two robot arms where one is remotely operated, while the other is autonomous and equipped with an eye-in-hand camera sensor. We propose a reactive unified convex optimization based controller that allows the execution of occlusion-free tasks by autonomously adjusting the camera robot, so as to keep the teleoperated one in the field of view. The occlusion avoidance is formulated inside the optimization as a constraint in the image space, it is formally derived from a collision avoidance analogy and made robust against noisy measurements and dynamic environment. A state machine is used to switch the control policy whenever an occlusion might occur. We validate our approach with experiments on a 14 d.o.f. dual-arm ABB YuMi robot equipped with an red, green, blue (RGB) camera and teleoperated by a 3 d.o.f. Novint Falcon device.