Hand Gesture Recognition-Based Control System Using Computer Vision for Transporter Robot

Abstract

Robotics is a discipline that has experienced rapid development in recent decades. One of the main focuses in robotics development is the advancement of control systems. The primary objective is to create control systems that are more natural, easily understood, easily operated by humans, thereby enabling more intuitive interactions between humans and robots. To achieve this goal, research based on hand gesture recognition has been conducted by adapting computer vision methods as the control system for controlling a transporter robot. In this paper, computer vision methods are employed using the image processing framework called MediaPipe. MediaPipe is used to recognize and comprehend hand gesture movements, as well as display hand landmarks. The research utilizes two hand landmark functions: “Finger Up” for the right hand to control the wheel movement, and “Range Gesture” for the left hand to control the arm and gripper of the transporter robot. The testing results indicate that “Finger Up” function for the right hand successfully controls robot transporter’s wheels with a success rate of 100%. Meanwhile, “Range Gesture” function for the left hand effectively controls the transporter robot’s arm and gripper with a success rate of 97%, with a total average error of 3%.