Optimal Control of a Two-Wheeled Self-Balancing Robot by Reinforcement Q-learning

Abstract

This paper concerns optimal control of the linear motion, tilt motion, and yaw motion of a two-wheeled self-balancing robot (TWSBR). Traditional optimal control methods for the TWSBR usually require a precise model of the system, and other control methods exist that achieve stabilization in the face of parameter uncertainties. In practical applications, it is often desirable to realize optimal control in the absence of the precise knowledge of the system parameters. This paper proposes to use a new feedback-based reinforcement Q-learning method to solve the linear quadratic regulation (LQR) control problem for the TWSBR. The proposed control scheme is completely online and does not require any knowledge of the system parameters. Both state feedback optimal control and output feedback optimal control are presented. Numerical simulation shows that the proposed optimal control scheme is capable of stabilizing the system and converging to the LQR solution obtained through solving the algebraic Riccati equation.