Learning Robot Manipulation based on Modular Reward Shaping

Abstract

Recently, reinforcement learning utilizing deep learning, called deep reinforcement learning, is being developed for discontinuous action space. It is shown that deep reinforcement learning has higher playing skills than those of human in the video game as Atari. The development of deep reinforcement learning for discontinuous action space, has been extended to the continuous action space as robot control area. The area of control deals with issues closely related to real world environment. However, deep reinforcement learning has limitations to solve control problems in the real world due to numerous trial-and-error based training. To address the limitations of learning in the real world, virtual world environments are used to emulate control problem in real world. In this paper, to solve a complex robot manipulation task, the learning robot manipulation based modular reward shaping is proposed. Since it is difficult to solve the complex task using only single primitive manipulation skill with a reward, it is required to combine various primitive manipulation skills with various rewards. In simulation results, it is shown that the combination of the reward functions affects to the robot manipulation performance.