A digital twin concept to prototype multi-drone based applications

Abstract

This article presents a digital twin that aims to serve as a prototyping platform for algorithms and services integrating Unmanned Aerial Vehicles (UAVs, a.k.a.drones).The digital twin enables to manage a real indoor "operation space", equipped with a cohort of aerial (drones) and ground robots, and an advanced tracking system.This is done from a virtual scene (Unity-based) that can be configured to simulate different application scenarios.Thus, real robots are connected into the virtual environment as IoT platforms.Bidirectional communication enables to both govern the physical space and receive feedback of physical changes into the virtual environment.In a demonstrative use-case, a surveillance and tracking mission for ground target detection in a security set up is assessed.Once the user configures the mission in the virtual tool, it is executed and tracked in the real environment.Connected drones provide their video input, which is processed for target detection and localization based on the simultaneous estimation of depth from the drones' monocular cameras.The digital twin concept enables to test architectures for 5G (e.g.on-board, edge,