Helping computer vision by verbal and nonverbal communication

Abstract

Proposes a method of removing ambiguities in robot tasks by a multimodal human-robot interface consisting of verbal and nonverbal communication. Such ambiguities often arise from failures of the robot vision system. However, it is not easy to solve this problem only by improving computer vision techniques. Thus, our robot asks a human such a question that a natural reply to it will contain helpful information to adapt the vision system for the current situation. We present a robot system that can bring the object ordered by a human by verbal and nonverbal behaviors.