Toward a One-interaction Data-driven Guide: Putting Co-speech Gesture Evidence to Work for Ambiguous Route Instructions

Abstract

Figure We highlight our proposed pipeline of capturing gestures through human example (motion capture), retargeting them to a robot, selecting the appropriate gesture for instruction depending on goal, selecting the best arm and standing formation to maximize understandability. This data-driven framework may allow for usable, readable, direction providing robots that can leverage ambiguous utterances. We show that each of these decisions have an impact on the usability of the synthesized instructional plan.