Deep Reinforcement Learning Based Co- Optimization of Morphology and Gait for Small-Scale Legged Robot

摘要

Small-scale legged robots have found widespread utilization in various industrial and biomedical applications due to their compact size and superior locomotion capabilities. Reducing the number of actuators is often desirable to decrease the robot's size and weight, which comes at the expense of the robot's workspace. Our study proposes a method to enhance the mobility of small-scale legged robots with limited degrees of actuators (DoAs) by co-optimizing both morphology parameters and control policy. The co-optimization is formulated as a bi-level optimization problem, where the control policy is designed using deep reinforcement learning algorithms and central pattern generators (CPGs) at the lower level. The inclusion of CPGs significantly speeds up training and enables the application of simulation results in real-world scenarios. At the upper level, morphology optimization is achieved through Bayesian optimization based on dual-networks. This approach eliminates the need to train a policy for each morphology candidate from scratch, leveraging previous experience to enhance efficiency. Through simulation and physical experiments, the effectiveness of our proposed approach is demonstrated, showcasing its ability to discover optimal morphology and gait for small-scale legged robots with limited DoAs. These findings have potential long-term impacts on small-scale legged robot design and locomotion control.