On the assessment of reinforcement learning techniques for trayectory tracking of autonomous ground robots

Abstract

This study investigates the application of Deep Reinforcement Learning (DRL) techniques in the trajectory tracking control of autonomous mobile robots. Assessing control performance under model parameter variations and disturbances, this work evaluates the suitability of DRL for a more adaptable and robust navigation solution. It was evaluated Deep Deterministic Policy Gradient (DDPG) and Twin-Delayed Deep Deterministic Policy Gradient (TD3) techniques to identify their strengths and limitations when using information of real-time dynamics, whereas a Model Predictive control (MPC) strategy is incorporated to contrast robust performance when using uncertain prediction models. The DDPG approach effectively learns complex policies for trajectory tracking but requires a careful balance between exploration and exploitation, and it has limited adaptability to environmental changes. In contrast, TD3 outperforms DDPG and NMPC in terms of trajectory tracking error and control effort across different model uncertainties and disturbances. This work highlights the potential for future research to enhance DRL-based controllers, particularly in handling uncertainties and disturbances during autonomous navigation tasks.