Joint Trajectory and Radio Resource Optimization for Autonomous Mobile Robots Exploiting Multi-Agent Reinforcement Learning

Abstract

Rapid and efficient sensor data acquisition plays a critical role in the decision-making process of each robot in a multi-robot smart factory. This paper investigates the trajectory design of autonomous mobile robots (AMRs) and communication resource allocation problems in industrial Internet of Things. Specifically, by exploiting both power and spatial domains, we adopt non-orthogonal multiple access to improve network connectivity in a spectrum-efficient manner, while the multi-antenna technique is employed to enhance diversity gain. The average sum rate is maximized by jointly optimizing the transmit power of sensors and the trajectory of AMRs. To deal with prior knowledge and dynamic channel conditions, we reformulate the long-term maximization problem as a Markov decision process, and further develop a provably efficient multi-agent reinforcement learning algorithm with a near-optimal regret bound. Our theoretical analysis reveals that both the decentralized execution and the experience exchange method are beneficial to accelerate convergence. Simulation results show that our proposed algorithm can reduce at least 80% convergence time compared to the centralized baseline, and can gain better rewards than the conventional <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\epsilon $ </tex-math></inline-formula> -greedy exploration.