Review on Peg-in-Hole Insertion Technology Based on Reinforcement Learning

Abstract

Peg-in-hole insertion is a critical process in industrial production. Traditional peg-in-hole insertion methods are based on planning the robot's motion trajectory through the analysis of contact models. However, due to the complexity of contact states, it's challenging to establish precise and reliable contact models, leading to poor generalization of these methods. Reinforcement learning is a technique that learns insertion strategies from environmental interactions, avoiding the tedious process of analytical modeling. Thus, it has become a trending direction in the robotics field in recent years. This article aims to survey the mainstream peg-in-hole insertion technologies based on reinforcement learning methods and discuss future research directions. First, we introduce the task requirements for peg-in-hole insertion. Subsequently, a preliminary framework of reinforcement learning algorithms for peg-in-hole insertion is presented. Discussions are then divided into two main categories: traditional reinforcement learning methods (including model-based and model-free methods) and reinforcement learning methods accelerated by prior knowledge (including residual reinforcement learning, reinforcement learning from demonstration, meta-reinforcement learning, and other acceleration techniques). Finally, this article explores several potential future research directions for peg-in-hole insertion technologies based on reinforcement learning.