Motion Planning and Obstacle Avoidance for Robot Manipulators Using Model Predictive Control-based Reinforcement Learning

Abstract

This paper presents a Nonlinear Model Predictive Control-based Reinforcement Learning (NMPC-based RL) framework for robot manipulators. The controller is developed to address the motion planning problem for robot manipulators in the presence of obstacles. The proposed control scheme includes a parametrized NMPC structure used as an approximator for the RL framework’s value function and action-value function. In the NMPC structure, the cost function, system constraints, and the manipulator’s model are parameterized. The Q-Learning algorithm based on the Temporal Difference method adjusts the parameters of the NMPC to increase the closed-loop performance of the whole control scheme. The controller has been applied to a 6-degrees-of-freedom (DoF) model of a robot manipulator, aimed at moving its end-effector to reach the desired pose when static obstacles are in the robot’s workspace. Numerical simulations demonstrate that the proposed controller can effectively control the end-effector’s pose in such a way as to avoid any collisions between the manipulator and the obstacles. It is shown that the learning capability of the proposed NMPC-based RL framework can enhance the efficiency of the control loop up to 21%.