“Do This Instead”—Robots That Adequately Respond to Corrected Instructions

Abstract

Natural language instructions are effective at tasking autonomous robots and for teaching them new knowledge quickly. Yet, human instructors are not perfect and are likely to make mistakes at times and will correct themselves when they notice errors in their own instructions. In this article, we introduce a complete system for robot behaviors to handle such corrections, during both task instruction and action execution. We then demonstrate its operation in an integrated cognitive robotic architecture through spoken language in two tasks: a navigation and retrieval task and a meal assembly task. Verbal corrections occur before, during, and after verbally taught sequences of tasks, demonstrating that the proposed methods enable fast corrections not only of the semantics generated from the instructions but also of overt robot behavior in a manner shown to be reasonable when compared to human behavior and expectations.