Adaptive State Space Formation Method for Reinforcement Learning.

摘要

For the application of reinforcement learning to real-world problems, an internal state space has to be constructed from a high dimensional observation space. The algorithm presented here constructs the internal state space during the course of learning desirable actions, and assigns local basis functions adaptively depending on the task requirement. The internal state space initially has only one basis function over the entire observation space, and that basis is eventually divided into smaller ones due to the statistical property of locally weighted temporal difference error. The algorithm was applied to an autonomous robot collision avoidance problem, and the validity of the algorithm was evaluated to show, for instance, the need of a smaller number of basis functions in comparison to other method.