Generating finely synchronized gesture and speech for humanoid robots: A closed-loop approach

Abstract

Previous work on the production of communicative robot gesture has not sufficiently addressed the challenge of speech-gesture synchrony. We propose a novel closed-loop multimodal scheduler that comprises two features to improve the synchronization process. First, the scheduler integrates an experimentally fitted forward model at the behavior planning stage to provide a more accurate estimation of the robot's gesture preparation time. Second, the model incorporates a feedback-based adaptation mechanism which allows for on-line adjustment of the synchronization of the two modalities during execution.