Point-Based Policy Synthesis for POMDPs With Boolean and Quantitative Objectives

摘要

Effectively planning robust executions under uncertainty is critical for building autonomous robots. Partially observable Markov decision processes (POMDPs) provide a standard framework for modeling many robot applications under uncertainty. We study POMDPs with two kinds of objectives: (1) Boolean objectives for a correctness guarantee of accomplishing tasks and (2) quantitative objectives for optimal behaviors. For robotic domains that require both correctness and optimality, POMDPs with Boolean and quantitative objectives are natural formulations. We present a practical policy synthesis approach for POMDPs with Boolean and quantitative objectives by combining policy iteration and policy synthesis for POMDPs with only Boolean objectives. To improve efficiency, our approach produces approximate policies by performing the point-based backup on a small set of representative beliefs. Despite being approximate, our approach maintains validity (satisfying Boolean objectives) and guarantees improved policies at each iteration before termination. Moreover, the error due to approximation is bounded. We evaluate our approach in several robotic domains. The results show that our approach produces good approximate policies that guarantee task completion.