Enriched manipulation action semantics for robot execution of time constrained tasks

Abstract

This paper contributes to semantic representation of human demonstrated actions for robot execution of time constrained tasks. We propose a semantic action encoding method based on interactions between the subject and objects in the scene. Our semantic framework is enriched with a descriptive spatial reasoning method which leads to accurate segmentation and recognition of unique action primitives. The proposed framework can classify the segmented action primitives as periodic or discrete to allow robots to autonomously decide how to imitate observed actions at different temporal scales. We evaluated our framework on our new large manipulation action dataset which involves in total 70 demonstrations from 8 different action types. We conducted various experiments with our humanoid robot to evaluate the proposed framework.