Baseline Policy Adapting and Abstraction of Shared Autonomy for High-Level Robot Operations

Abstract

This paper presents a novel shared autonomy and baseline policy adapting framework for human-robot interactions in high-level context-aware robotic tasks. With a unique methodology that leverages hierarchies in decision-making as well as variational analysis of human policy, we propose a mathematical model of shared autonomy policy. The framework aims at interpretable high-level decision-making for efficient robot operation with human in the loop. We modeled the decision-making process using hierarchical Markov decision processes (MDPs) in an algorithm we called <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">policy adapting</i>, where the autonomous system policy is adapted, and hence, shaped by incorporating design variables contextual to the robot, human, task, and pre-training. By integrating deep reinforcement learning within a multi-agent hierarchical context, we present an end-to-end algorithm to train a baseline policy designed for shared autonomy. We showcase the effectiveness of our framework, and particularly the interplay between different design elements and human's skill level, in a pilot study with a human user in a simulated sequence of high-level pick-and-place tasks. The proposed framework advances the state-of-the-art in shared autonomy for robotic tasks, but can also be applied to other domains of autonomous operation.