Reinforcement Learning in Multi-dimensional State-action Space Using Random Tiling and Gibbs Sampling

Abstract

In real-robot applications, learning controllers are often required to obtain control rules over high-dimensional continuous state-action space. Random tile-coding is a promising method to deal with high-dimensional state space for representing the state value function. However, there is no standard reinforcement learning scheme to deal with action selection in high-dimensional action space, especially the probability of action variables are mutually dependent. This paper introduces a new action selection scheme using random tile-coding and Gibbs sampling, and shows the Q-learning algorithm applying the proposed scheme. We demonstrate it through a Rod in maze problem and a redundant arm reaching task.