Multi-performance index reinforcement learning training of beaver-like robot

Abstract

Abstract The underwater environment is characterized by its inherent complexity and dynamics, leading to substantial interference with the precision of underwater measurement data. To enhance the precision of underwater data measurements, underwater robotic platforms necessitate improved motion and stability characteristics. As amphibian mammals, beavers possess excellent amphibious abilities and a wide range of environmental adaptability. Based on the observation of biological morphology and hind limb fin structure of beavers, this study analyses their swimming mechanism and designs a beaver-like robot. This study introduces an efficient control algorithm designed for a beaver-like robot platform. The algorithm integrates reinforcement learning with conservative Q-learning, model-based policy optimization and deep Q-network methods to facilitate offline training of the robot. A training weight allocation system is employed to enhance adaptability across diverse swimming conditions in the complex underwater setting. Simulating the robot’s underwater environment, the algorithm has demonstrated effective training in both speed and stability. The pitch angle is successfully stabilized between −0.245 and 0.305 rad, while the robot’s speed reaches up to 0.38 m·s −1 .