Decomposed Deep Reinforcement Learning for Robotic Control

Abstract

We study how structural decomposition and interactive learning among multiple agents can be utilized by deep reinforcement learning in order to address high dimensional robotic control problems. We decompose the whole control space of a certain robot into multiple independent agents according to this robot's physical structure. We then introduce the concept of Degree of Interaction (DoI) to describe the level of dependencies (i.e., the necessity of coordination) among the learning agents. Three different methods are then proposed to compute the DoI dynamically during learning. The experimental evaluation demonstrates that the decomposed learning method is substantially more sample efficient than the state-of-the-art algorithms, and more explicit interpretations can be generated on the final learned policy as well as the underlying dependencies among the learning agents.