Learning Local Optimal Controller for a Class of Nonlinear Systems via Impulse-Supervised Exploration

Abstract

This paper develops an impulse-supervised confined exploration framework for learning local optimal controller for a class of nonlinear systems. The proposed approach combines continuous-time approximate dynamic programming (ADP) with an impulsive supervisory layer, where impulsive braking confines the state within a prescribed region in which a local linear approximation of the nonlinear system is valid. This enables desired persistent excitation required for parameter convergence while preventing large state deviations that invalidate local optimality. The resulting hybrid closed-loop system enforces invariance of the exploration region through state-triggered braking inputs. Simulation results on a nonlinear mechanical system demonstrate effectiveness of the proposed approach.