Sequential alternating least squares for solving high dimensional linear Hamilton-Jacobi-Bellman equation

Abstract

This paper presents a technique to efficiently solve the Hamilton-Jacobi-Bellman (HJB) equation for a class of stochastic affine nonlinear dynamical systems in high dimensions. The HJB solution provides a globally optimal controller to the associated dynamical system. However, the curse of dimensionality, commonly found in robotic systems, prevents one from solving the HJB equation naively. This work avoids the curse by representing the linear HJB equation using tensor decomposition. An alternating least squares (ALS) based technique finds an approximate solution to the linear HJB equation. A straightforward implementation of the ALS algorithm results in ill-conditioned matrices that prevent approximation to a high order of accuracy. This work resolves the ill-conditioning issue by computing the solution sequentially and introducing boundary condition rescaling. Both of these additions reduce the condition number of matrices in the ALS-based algorithm. A MATLAB tool, Sequential Alternating Least Squares (SeALS), that implements the new method is developed. The performance of SeALS is illustrated using three engineering examples: an inverted pendulum, a Vertical Takeoff and Landing aircraft, and a quadcopter with state up to twelve.