Real-Time Decision Making of Autonomous Robot under Uncertainty of State Estimation by Using Particle Filter and Q-MDP Value Method

Abstract

We propose the real-time Q-MDP value method for decision making of a robot under uncertain state recognition. When the computation result of a control problem is known on the assumption that recognition is certain, the original Q-MDP value method decides an appropriate action based on uncertain recognition. The method is not suitable for real-time decision making due to the complexity of probability calculation. In the real-time Q-MDP value method, a particle filter that is utilized for state estimation is directly used for the probability calculation. The proposed method can make it possible to execute the Q-MDP value method in real-time. The proposed method is applied to total behavior of a goalkeeper for robot soccer competition. Experiments and actual games have suggested that this method can decide actions effectively according as uncertain result of state estimation.