Sim-to-Real Visual Grasping via State Representation Learning Based on Combining Pixel-Level and Feature-Level Domain Adaptation

Abstract

In this study, we present a method to grasp diverse unseen real-world objects using an off-policy actor-critic deep reinforcement learning (RL) with the help of a simulation and the use of as little real-world data as possible. Actor-critic deep RL is unstable and difficult to tune when a raw image is given as an input. Therefore, we use state representation learning (SRL) to make actor-critic RL feasible for visual grasping tasks. Meanwhile, to reduce visual reality gap between simulation and reality, we also employ a typical pixel-level domain adaptation that can map simulated images to realistic ones. In our method, as the SRL model is a common preprocessing module for simulated and real-world data, we perform SRL using real and adapted images. This pixel-level domain adaptation enables the robot to learn grasping skills in a real environment using small amounts of real-world data. However, the controller trained in the simulation should adapt to the real world efficiently. Hence, we propose a method combining a typical pixel-level domain adaptation and the proposed SRL model, where we perform SRL based on a feature-level domain adaptation. In evaluations of vision-based robotics grasping tasks, we show that the proposed method achieves a substantial improvement over a method that only employs a pixel-level or domain adaptation.