Seeing Through Walls: Real-Time Digital Twin Modeling of Indoor Spaces

Abstract

As the need for situational awareness rises in search and rescue tasks, systems for human spatial sensing augmentation in complex-built environments has become increasingly important. Sensor-based mapping and augmented reality (AR) techniques have been tested to provide enhanced visual assistance in indoor scenarios, such as “seeing through walls” abilities for occluded areas. Scholars have examined various scanning techniques and UI/UX designs for AR-based occlusion visualization. However, evidence is still needed to show how scene capturing, 3D mapping, rendering and visualization can be executed integratively, enabling an instantaneous “Digital Twins” modeling. This paper presents a robot-based system for mapping indoor environments and creating a near real-time virtual replica. A dynamically updated 3D map rendered with Unity generates egocentric “x-ray vision” views of the occluded objects using AR headsets. A case study was performed in a lab room, resulting in virtual regeneration of surroundings supporting search and rescue in occluded areas.