Towards Learning to Detect and Predict Contact Events on Vision-based\n Tactile Sensors

Abstract

In essence, successful grasp boils down to correct responses to multiple\ncontact events between fingertips and objects. In most scenarios, tactile\nsensing is adequate to distinguish contact events. Due to the nature of high\ndimensionality of tactile information, classifying spatiotemporal tactile\nsignals using conventional model-based methods is difficult. In this work, we\npropose to predict and classify tactile signal using deep learning methods,\nseeking to enhance the adaptability of the robotic grasp system to external\nevent changes that may lead to grasping failure. We develop a deep learning\nframework and collect 6650 tactile image sequences with a vision-based tactile\nsensor, and the neural network is integrated into a contact-event-based robotic\ngrasping system. In grasping experiments, we achieved 52% increase in terms of\nobject lifting success rate with contact detection, significantly higher\nrobustness under unexpected loads with slip prediction compared with open-loop\ngrasps, demonstrating that integration of the proposed framework into robotic\ngrasping system substantially improves picking success rate and capability to\nwithstand external disturbances.\n