Microwave Reconstruction of 3-D Human Facial Landmarks Using a Programmable Metasurface

Abstract

Nowadays, 3D human facial reconstruction (FR) plays a pivotal role in various areas including human-robot interaction, virtual reality, identification, and so on. However, conventional FR strategies are largely based on optical sensors, and thus suffer from the intrinsic risk in terms of obstacle (e.g., mask) occlusion, light-condition dependence, privacy infringement, and so on. To tackle it, we here propose to leverage the microwave for the 3D FR, by utilizing a fundamental fact that the microwave, unlike optics, is capable of operating in the conditions of all-weather, all-day and occlusion. To this end, we design an inexpensive transceiver architecture based on a largeaperture programmable metasurface, enabling to capture fully the 3D facial information of the subject, meanwhile suppressing the unwanted disturbances from surrounding environment and other body parts. Afterwards, we design a deep artificial network for converting the microwave raw data into the 3D coordinates of 68 facial landmarks. We fabricate a proof-of-principle prototype system working within the frequency range of 25–26.5 GHz, and experimentally demonstrate that the reported strategy is efficient in the reconstruction of 3D human facial landmarks even for the subject wearing a mask