A Control Method of Robotic Arm Based on Improved Deep Deterministic Policy Gradient

Abstract

With the continuous development of Machine Learning, Reinforcement Learning has achieved excellent results in many fields. An improved Deep Deterministic Policy Gradient (DDPG) algorithm is proposed in this paper to optimize Reinforcement Learning control performance, because of the issue of sparse rewards during training in continuous action spaces such as robotic arms, leading to slow convergence and low success rates. The algorithm merges the DDPG algorithm with the Hindsight Experience Relay (HER) algorithm, enabling the agent to learn from the failed experience when learning. Moreover, a double experience replay buffer is introduced, comprising a primary buffer and a secondary buffer containing high-reward experiences, which enhances the sampling probability of high-quality samples. At long last, a PyBullet simulation environment was utilized to execute a simulation experiment. The robotic arm using the improved DDPG algorithm achieved a success rate of nearly 100% in about 220 epochs of training, and the algorithm convergence speed was greatly improved. The trained robotic arm can reach any target point in the workspace. The efficacy of the improved DDPG algorithm is demonstrated by the outcomes, providing a valuable reference for further exploration into the intelligent control of robotic arms for object grasping.