Air-ground robot team surveillance of complex 3D environments

Abstract

Real-world surveillance of complex 3D environments is an extremely challenging problem, especially in the presence of unknown dangers. In this work we create a novel ground and aerial autonomous robotic system to surveil a human-robot team's surroundings for targets of interest, which could be e.g., disaster victims, infrastructure inspection, data to support research or public safety, or threats to the human team members' safety. To represent this general case, our system identifies threats to human safety by surveilling the team's surroundings, identifying threats, and notifying the human team member. We provide an interface that visualizes threat targets and allows the human operator to create and modify the surveillance plan. We note that this 3D surveillance task resembles the Art Gallery Problem (AGP) with a time-sensitive route planning aspect similar to the Traveling Salesman Problem (TSP), both of which are NP-hard. We incorporate a human operator into the decision making process of a surveillance system to address the viewpoint selection and route minimization problems, and to extract semantic information from the scene to increase search effectiveness. We construct a system for this collaborative, human-robot team surveillance task using a low-cost Unmanned Aerial Vehicle (UAV) and a more-capable Unmanned Ground Vehicle (UGV). We evaluate the resulting system with a large experiment set (120 trials) conducted in a real-world, 3D, cluttered, urban setting and examine the difference a scenario-specific plan makes to the detection of threat targets compared to a baseline algorithmically-generated plan.