A Novel Adaptive Actor-Critic Reinforcement Learning Controller for Constrained Robots

Abstract

Force-position control for robot manipulators has become a basic control regime for advanced applications such as interaction or cooperative tasks that imply contact to environment or external object. Thus, for many applications environmental information is evaluated continuously to generate adequate control actions to complete the task. There has been proposed techniques such as adaptive or intelligent control to cover some of these requirements, however it seems insufficient given the uncertainty involved when dealing with the environment. The emergence of deep learning has recently renewed interest in these tasks from the perspective of reinforcement learning algorithms. In this paper, an actor-critic scheme is proposed for simultaneous control of contact force and joint positions while moving along an environmental rigid surface. The critic stage evaluates information of the environment through an extended error manifold, trying to minimize the cumulative future rewards by a neural network. The actor stage, which receives environment information from the critic stage, is used to approximate inverse robot dynamics. Together with two continuous orthogonal sliding PID controllers gives rise to simultaneous tracking of position and force, with stability analysis. Numerical simulations show the performance of the proposed approach under several conditions.