A Study of Q-learning: Dynamic Structuring of Exploration Space Based on Genetic Algorithm

Abstract

Reinforcement learning has recently received much attention as a learning method for complicated systems, e.g., robot systems. It does not need prior knowledge and has higher capability of reactive and adaptive behaviors. However increase in dimensionality of the action-state space makes it diffcult to accomplish learning. The applicability of the existing reinforcement learning algorithms are effective for simple tasks with relatively small action-state space. In this paper, we propose a new reinforcement learning algorithm: “Q-learning with Dynamic Structuring of Exploration Space Based on Genetic Algorithm ”. The algorithm is applicable to systems with high dimensional action and interior state spaces, for example a robot with many redundant degrees of freedom. To demonstrate the effectiveness of the proposed algorithm simulations of obstacle avoidance by a 50 links manipulator have been carried out. It is shown that effective behavior can be learned by using the proposed algorithm.