Learning to Augment Synthetic Images for Sim2Real Policy Transfer

Abstract

Vision and learning have made significant progress that could improve\nrobotics policies for complex tasks and environments. Learning deep neural\nnetworks for image understanding, however, requires large amounts of\ndomain-specific visual data. While collecting such data from real robots is\npossible, such an approach limits the scalability as learning policies\ntypically requires thousands of trials. In this work we attempt to learn\nmanipulation policies in simulated environments. Simulators enable scalability\nand provide access to the underlying world state during training. Policies\nlearned in simulators, however, do not transfer well to real scenes given the\ndomain gap between real and synthetic data. We follow recent work on domain\nrandomization and augment synthetic images with sequences of random\ntransformations. Our main contribution is to optimize the augmentation strategy\nfor sim2real transfer and to enable domain-independent policy learning. We\ndesign an efficient search for depth image augmentations using object\nlocalization as a proxy task. Given the resulting sequence of random\ntransformations, we use it to augment synthetic depth images during policy\nlearning. Our augmentation strategy is policy-independent and enables policy\nlearning with no real images. We demonstrate our approach to significantly\nimprove accuracy on three manipulation tasks evaluated on a real robot.\n