Adaptive Safety-Certified Reinforcement Learning for Constrained Optimal Control of Autonomous Robots With Uncertainties

Abstract

This paper investigates a constrained optimal control problem for safety-critical robots with parametric uncertainties. A novel adaptive safety-certified reinforcement learning (RL) algorithm is proposed, leveraging control barrier functions (CBFs) to enable safe learning of the optimal policy during the online exploration phase. Specifically, a high-order robust adaptive CBF is presented to minimally adjust RL-derived control actions by incorporating a prescribed-time adaptation law to handle the unknown system parameters. This way directly enforces forward invariance, allowing the shrunken safe set to near the standard set within a user-prescribed time. Moreover, a novel adaptive critic learning frame is presented by introducing filtered auxiliary signals that integrate both instantaneous and historical data, which relaxes the strict persistent excitation (PE) condition required in the existing RL methods to a weaker, easily verifiable finite excitation (FE) condition. Later, a prescribed-time learning rule is developed to accelerate the convergence of weights. The key advantage of the proposed way is the decoupling of safety and RL convergence, enabling each component to be managed separately, thereby offering stronger safety certifications compared to the existing RL schemes even under uncertain dynamics. The effectiveness and superiority of the proposed scheme are proven via simulations for surveillance and regulation tasks of autonomous robots.