Multifingered Object Recognition with Tactile Sensors and Graph Convolutional Networks using Topological Graph Segmentation

Abstract

This study investigates the application of topological segmentation to Graph Convolutional Networks (GCNs) for object recognition using tactile data from a multi-fingered robotic hand. While GCNs have shown promise in processing tactile information, the large volume of data from distributed tactile sensors poses challenges. Inspired by neurological research indicating intra-digit segmentation in human hand topology, we propose two methods of topological segmentation for GCNs: segmenting by digits and palm, and segmenting by individual skin patches. We evaluate these methods against a non-segmented GCN baseline using various input modalities including tactile features, taxel positions, and joint angles. Data was collected from an Allegro Hand equipped with uSkin tactile sensors, manipulating eight everyday objects. Our results demonstrate that topological segmentation enhances object recognition performance, with the best model achieving a 92.92% recognition rate using patch-level segmentation and tactile features with joint angles as input. UMAP analysis of GCN features reveals that segmentation methods produce distinct representations for each hand segment. Additionally, topological segmentation significantly reduces computational resource requirements compared to non-segmented GCNs. This study contributes the first application of topological segmentation to GCNs for tactile processing in robotic hands, achieving high object recognition rates and providing insights into feature extraction capabilities. The proposed method shows potential for improving efficiency and performance in tactile-based robotic manipulation tasks.