Affordance-based Actionable Semantic Mapping and Planning for Mobile Rescue Robots

Abstract

Autonomous and tele-operation of rescue robots in urban search and rescue (USAR) environments is very challenging as details of missions and environments are usually unknown, mission goals might change dynamically and there is only little repeatability between different missions. Therefore, we propose a novel actionable semantic mapping and planning approach which leverages complementary capabilities of operator and robotic assistance functions. While related methods often focus on accuracy for geometric or semantic representations, we propose a novel framework focusing on an actionable map representation which is well suited for planning complex behaviors in uncertain environments. We represent the environment topologically as a scene graph coupled with a geometrically and semantically dense representation as Truncated Signed Distance Functions. We propose to apply the concept of affordances to map possible actions and costs to object classes. Building on that, we propose a combined topological and geometric task planning method allowing for easy operator interaction on task selection and prioritization. The successful application in two complex scenarios demonstrates the flexibility and efficiency of the proposed approach and the benefit of operator interaction.