An integrated vision-based system for efficient robot arm teleoperation

Abstract

Purpose This paper aims to present a natural human–robot teleoperation system, which capitalizes on the latest advancements of monocular human pose estimation to simplify scenario requirements on heterogeneous robot arm teleoperation. Design/methodology/approach Several optimizations in the joint extraction process are carried on to better balance the performance of the pose estimation network. To bridge the gap between human joint pose in Cartesian space and heterogeneous robot joint angle pose in Radian space, a routinized mapping procedure is proposed. Findings The effectiveness of the developed methods on joint extraction is verified via qualitative and quantitative experiments. The teleoperation experiments on different robots validate the feasibility of the system controlling. Originality/value The proposed system provides an intuitive and efficient human–robot teleoperation method with low-cost devices. It also enhances the controllability and flexibility of robot arms by releasing human operator from motion constraints, paving a new way for effective robot teleoperation.