Residual-gradient-based neural reinforcement learning for the optimal control of an acrobot

Abstract

Based on the idea of dynamic programming, reinforcement learning (RL) has become an important model-free method to solve difficult optimal control problems. In this paper, a novel neural RL method is proposed to solve the time-optimal control problem of a class of under-actuated robots, which is called the acrobot. The RL method uses a modified residual gradient reinforcement learning algorithm called RGNP (residual gradient with nonstationary policy). The RGNP algorithm not only has guaranteed convergence under certain conditions but also can ensure the performance of the approximated optimal policy, which is superior to the previous residual gradient algorithms. Simulation results of the learning control of the acrobot illustrate the effectiveness of the proposed method.