Vision-Based Humanoid Robotic Palm for Amputee Person using Deep Learning

Abstract

Amputees often have to face certain challenges due to their physical limitations. This naturally affects their work and have to encounter issues of unemployment. Overcoming people's perspective of an amputee not working is something that the proposed system wishes to change. Thus, making the person self reliable and independent. This paper introduces a Humanoid Robotic Palm designed to assist amputees, enhancing their independence and quality of life. YOLOv5, VGG16, and ResNet architectures of CNN are used to train the model using a custom dataset of 2,667 annotated images featuring common objects, and that model is integrated with robotic palm for real-time object detection and classification. The Humanoid Robotic Palm is programmed to grasp the object by forming a grip according to the detected object. The Robotic Palm is controlled using Arduino Uno. PyFirmata Library is used to connect the Deep learning model with the Arduino Uno microcontroller. The accuracy achieved is 98.92% using ResNet Architecture, 96.55% using VGG16 Architecture, and 84.90% using YOLOv5.