Path Planning of Humanoid Arm Based on Deep Deterministic Policy Gradient

Abstract

The robot arm with multiple degrees of freedom and working in a 3D space needs to avoid obstacles during the grasping process by its end effector. Path planning to avoid obstacles is very important for accomplishing a grasping task. This paper proposes a new obstacle avoidance algorithm, based on an existing deep reinforcement learning framework called deep deterministic policy gradient (DDPG). Specifically, we propose to use DDPG to plan the trajectory of a robot arm to realize obstacle avoidance. The rewards are designed to overcome the difficulty in convergence of multiple rewards, especially when the rewards are antagonistic with respect to each other. Obstacle avoidance of the robot arm using DDPG is achieved by self-learning, and the convergence problem caused by the high dimension state input and multiple return values is solved. The simulation model of an arm of the Nao robot is built based on the MuJoCo simulation environment. The simulation demonstrates that the proposed algorithm successfully allows the robot arm to avoid obstacles.