Kernel Recursive Least Squares Function Approximation in Game Theory Based Control

Abstract

A game theoretic aspect in reinforcement learning based controller design with kernel recursive least squares algorithm for value function approximation is proposed in this paper. A kernel recursive least-squares-support vector machine is used to realize a mapping from state, controller's action and disturber's action to Q-value function. Online sparsification framework permits the addition of training sample into the Q-function approximation only if it is approximately linearly independent of the preceding training samples. Markov game setup is shown to be one of the important platforms for addressing robustness of direct adaptive optimal control of nonlinear systems. A game against nature strategy shows the strength of state importance in terms of accelerated learning, and better relative stability of the system. Simulation results on two-link robot manipulator show that the proposed method has high learning efficiency—better accuracy measured in terms of mean square error; and lesser computation time, compared to the least-squares support vector machine.