Design of Biped Robot Using Reinforcement Learning and Asynchronous Actor-Critical Agent (A3C) Algorithm

Abstract

The creation of a humanoid robot necessitates a remarkable interdisciplinary effort spanning engineering, mathematics, software, and machine learning. In this work, we investigate the policy-based algorithm known as Reinforce, which is a deep reinforcement method. The goal of policy-based approaches is to directly optimize the policy without the utilizes of a value function. Reinforce specifically belongs to the Policy-Gradient techniques subclass of Policy-Based techniques. This subclass uses gradient ascent to estimate the weights of the ideal policy, directly optimizing the policy. In order to stabilize the training by lowering the variance, a hybrid architecture combining policy-based and value-based methodologies is proposed in this paper. Asynchronous Advantage Actor-Critic (A3C), a hybrid technique, trains agents in robotic environments by employing Stable-Baselines3. It trains two agents to walk, one on two legs and the other on a spider moment. According to the experimental findings, both robots are able to recognize the target's orientation, move to the proper location, and then successfully raise the target together.