Decoupling feature extraction from policy learning: assessing benefits\n of state representation learning in goal based robotics

Abstract

Scaling end-to-end reinforcement learning to control real robots from vision\npresents a series of challenges, in particular in terms of sample efficiency.\nAgainst end-to-end learning, state representation learning can help learn a\ncompact, efficient and relevant representation of states that speeds up policy\nlearning, reducing the number of samples needed, and that is easier to\ninterpret. We evaluate several state representation learning methods on goal\nbased robotics tasks and propose a new unsupervised model that stacks\nrepresentations and combines strengths of several of these approaches. This\nmethod encodes all the relevant features, performs on par or better than\nend-to-end learning with better sample efficiency, and is robust to\nhyper-parameters change.\n