Using pressure sensors to identify manipulation actions during human physical interaction

Abstract

This paper presents an investigation of human physical interaction. In particular, we describe how data from pressure sensors mounted on a glove worn by a human can be mapped to manipulation actions; the actions can in turn be used to interpret physical interaction during elderly care. The work is part of the RoboHelper project, which aims to build a multimodal communication interface for assistive robots for the elderly. Human-human physical interaction during elderly care and in a realistic setting is studied in this work with the aim of using the learned insights to develop corresponding robot interfaces. The contribution of this work is the identification of various types of physical manipulation actions that take place when an elder is assisted in performing activities of daily living in a natural setting. As part of the RoboHelper project, it has been shown that the knowledge of actions involving physical manipulation of objects helps in understanding the spoken language. More specifically, it improves the resolution of third person pronouns/deictic words and the classification of dialogue acts. In this work we show that pressure sensor data can be used to automatically recognize such physical manipulation actions. The automatic recognition of physical manipulation actions may facilitate future studies of multimodal interaction by greatly reducing the time required for manual annotations. It is also useful for learning from demonstration, a popular approach in human-robot interaction research.