IMPROVING THE PERFORMANCE OF Q-LEARNING WITH LOCALLY WEIGHTED REGRESSION

Abstract

Oftentimes, the problem faced by researchers applying reinforcement learning to a nontrivial robotics problem is that they run head-on into the curse of dimensionality. This is a particular problem for those researchers using discrete-state algorithms, as the number of states exponentially increase with the complexity of the problem. This thesis provides a method by which the performance of a discrete-state algorithm can be improved when applied to a continuous-state problem in combination with a function approximator. The method consists of two steps. The first step consists of learning the value function over a small number of discrete states. The second step involves using the function approximator to generalize from those discrete states to a continuous state space.