Integration of Cell-Mapping and Reinforcement-Learning Techniques for Motion Planning of Car-Like Robots

摘要

<para xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> The aim of this work has been to integrate the Cartesian space together with the kinematics and dynamics spaces of a car-like robot. We propose a new algorithm that obtains a minimum-time solution to the optimal motion planning of the vehicle. The new algorithm is based on the combination of cell-mapping and reinforcement-learning techniques. This algorithm can obtain the environment and vehicle parameters from received experience without needing a mathematical model. The algorithm uses a transformation of the cell-to-cell transitions to reduce the time that is spent in the knowledge of the vehicle dynamics and environment. Four state variables have been considered: 1) the velocity of the vehicle; 2) the <formula formulatype="inline"><tex Notation="TeX">$x$</tex></formula> Cartesian coordinate; 3) the <formula formulatype="inline"><tex Notation="TeX">$y$</tex></formula> Cartesian coordinate; and 4) the orientation of the vehicle. In addition, two different control actions can act on the vehicle: 1) the traction torque that was used for speeding up/braking the vehicle and 2) the steering angle. The results show the applicability of the proposed algorithm in environments with the presence of obstacles. </para>