A soft thumb-sized vision-based sensor with accurate all-round force\n perception

Abstract

Vision-based haptic sensors have emerged as a promising approach to robotic\ntouch due to affordable high-resolution cameras and successful computer-vision\ntechniques. However, their physical design and the information they provide do\nnot yet meet the requirements of real applications. We present a robust, soft,\nlow-cost, vision-based, thumb-sized 3D haptic sensor named Insight: it\ncontinually provides a directional force-distribution map over its entire\nconical sensing surface. Constructed around an internal monocular camera, the\nsensor has only a single layer of elastomer over-molded on a stiff frame to\nguarantee sensitivity, robustness, and soft contact. Furthermore, Insight is\nthe first system to combine photometric stereo and structured light using a\ncollimator to detect the 3D deformation of its easily replaceable flexible\nouter shell. The force information is inferred by a deep neural network that\nmaps images to the spatial distribution of 3D contact force (normal and shear).\nInsight has an overall spatial resolution of 0.4 mm, force magnitude accuracy\naround 0.03 N, and force direction accuracy around 5 degrees over a range of\n0.03--2 N for numerous distinct contacts with varying contact area. The\npresented hardware and software design concepts can be transferred to a wide\nvariety of robot parts.\n