Deep Reinforcement Learning Based Brachiation Control for Two-Link Bio-Primate Robot

摘要

Manually designing an effective and efficient controller for complex mechanics, such as bio-inspired robots or underactuated mechanical system, typically are very difficult. It requires precise motion planning and dynamic control. Reinforcement learning or genetic algorithm based learning methods suffers from representing the high dimensional models. The combination of deep learning and reinforcement learning provide a feasible way to handle such difficulties. However, priori-less searching sometimes tends to be low efficient and usually finds the “mechanic” solution instead of the “natural” one. In this paper, the traditional nonlinear control concept is integrated into the deep reinforcement learning (DRL) framework. The whole process is implemented on the brachiation control problem of a two link bio-primate robot. Deep Deterministic Policy Gradient (DDPG) is used to search for the optimal control policy. The searching process is realized by interacting with the dynamic model instead of real robot. The energy based planning and control concept is adopted, which utilize the fact that when the shoulder joint angle is fixed, energy of the whole system keeps constant. By regulating the angle and energy, the robot can be restricted on a particular trajectory. The energy concept is encoded within the reward function and trained in the Gym environment. For varying targets point-to-point control, the network structure is also modified to accept the target coordinates. Effectiveness of the proposed methods are verified by simulation and experimental results.