The Role of the Trainer in Reinforcement Learning

摘要

In this paper we propose a three-stage incremental approach to the development of autonomous agents. We discuss some issues about the characteristics which differentiate reinforcement programs (RPs), and define the trainer as a particular kind of RP. We present a set of results obtained running experiments with a trainer which provides guidance to the AutonoMouse, our mouse-sized autonomous robot. 1 THE THREE STAGES OF THE DEVELOPMENTAL APPROACH Reinforcement learning (RL) problems are those problems in which an agent has the goal of learning how to maximize a scalar return which is functionally related to the agent's actions. Among the most studied and best known algorithms used by researchers in trying to solve RL problems there are Q-learning (Watkins, 1989), the adaptive heuristic critic (Barto, Sutton and Watkins, 1990), and the learning classifier system (Booker, Goldberg, Holland, 1989), which was used as the learning paradigm in the experiments of this paper. An issue we have...