Real-Time Decision Making with State-Value Function under Uncertainty of State Estimation – Evaluation with Local Maxima and Discontinuity

Abstract

We have proposed the real-time QMDP method for decision making of a robot under uncertain state recognition. This method evaluates every action and chooses the best one with a particle filter for estimation and a state-value function of dynamic programming. Different from our past work, this paper applies it to a complicated decision making task that yields local maxima and discontinuity on the state-value function. We then verify whether the method can choose proper actions or not in such a condition. As an example, total behavior of a goalkeeper for robot soccer is planned by using value iteration. This task contains three strategies, which are related to three kinds of local maxima respectively. Simulations, experiments and actual games have suggested that the method can decide actions effectively according as uncertain result of state estimation.