Deep Learning-Based 3D Pose Reconstruction of an Underwater Soft Robotic Hand and Its Biomimetic Evaluation

Abstract

Soft robotic hand shows considerable promise for various grasping applications. However, the sensing and reconstruction of the robot pose will cause limitation during the design and fabrication. In this work, we present a novel 3D pose reconstruction approach to analyze the grasping motion of a bidirectional soft robotic hand using experiment videos. The images from top, front, back, left and right views were collected using an one-camera-multiple-mirror (OCMM) imaging device. The experiment videos about underwater grasp were captured and further analyzed. The coordinate and orientation information of soft fingers are detected based on deep learning methods. A pretrained Faster RCNN model is used to detect the position of fingertips, while a pretrained U-Net classifier is applied to determine the side shape of the fingers. Based on the kinematic modelling, the corresponding coordinate and orientation databases are established. The 3D pose reconstructed results present a satisfactory performance and effectiveness. Based on efficacy coefficient method, the finger contribution of the bending angle and distance between fingertips of soft robotic hand was analyzed compared with that of human hand. The results show that the soft robot hand performs a human-like motion in both single-direction and bidirectional grasping.