Multirobot Navigation Using Partially Observable Markov Decision Processes with Belief-Based Rewards

Abstract

For the safe deployment of multirobot systems, accurate position estimation is necessary. We present a multirobot navigation algorithm that accomplishes the dual objective of reaching designated goal positions and maintaining low position uncertainty. We pose the problem as a [Formula: see text]-partially observable Markov decision process ([Formula: see text]POMDP) to capture position uncertainty through a belief-based reward. The multirobot system employs a centralized extended Kalman filter (EKF) for state estimation, whose output is used as the observation as well as the belief in the [Formula: see text]POMDP. Using the EKF output as the observation and belief allows for the integration of the EKF in the action planning module of the system. To solve the [Formula: see text]POMDP, we adapt the online information particle filter tree algorithm to be compatible with the EKF closed-form output being used as both the observation and belief. We present results for the proposed navigation algorithm that show emergent behaviors for a multirobot system, where the robots move to provide inter-ranging measurements that minimize position uncertainty for all robots in the system. We also demonstrate that the proposed method outperforms a state-of-the-art solver (partially observable Monte Carlo planning with observation widening) and a direct-to-goal planner in terms of minimizing position uncertainty.