Improved Switching among Temporally Abstract Actions

Abstract

Actions Richard S. Sutton Department of Computer Science University of Massachusetts Amherst, MA 01003-4610 [email protected] Satinder Singh Department of Computer Science University of Colorado Boulder, CO 80309-0430 [email protected] Doina Precup Department of Computer Science University of Massachusetts Amherst, MA 01003-4610 [email protected] Balaraman Ravindran Department of Computer Science University of Massachusetts Amherst, MA 01003-4610 [email protected] Abstract In robotics and other control applications it is commonplace to have a preexisting set of controllers for solving subtasks, perhaps handcrafted or previously learned or planned, and still face a difficult problem of how to choose and switch among the controllers to solve an overall task as well as possible. In this paper we present a framework based on Markov Decision Processes and Semi-Markov Decision Processes for phrasing this problem, a basic theorem regarding the improvement in performance that ...