Integrating POMDP and reinforcement learning for a two layer simulated robot architecture

Abstract

Two layer control systems are common in robot architectures. The lower level is designed to provide fast, fine grained control while the higher level plans longer term sequences of actions to achieve some goal. Our approach uses reinforcement learning (RL) for the low level and Partially Observable Markov Decision Process (POMDP) planning for the high level. Because both levels can adapt their behavior within the scope of their tasks, the combination is expected to be robust to degradations in sensor and actuator failures and so to enhance overall system reliability. We implemented our architecture for use in the Khepera robot simulator. In a set of experiments, we show that good performance can be difficult to achieve with hand coded low level control and that performance of our RL/POMDP system degrades slowly with increasing sensor and actuator failure. 1 Introduction Two layer control systems are common in robot architectures. Two layer control is a pragmatic solution to different...