Indoor Layout Estimation by Fusing Monocular RGB Image Features Extracted with HRNet

Abstract

Indoor layout estimation (ILE) is a challenging research that segments the natural indoor scene image into floor, walls and ceiling, and it has a wide range of application prospects in the fields such as scene understanding, reconstruction, robot positioning, and virtual reality. An ILE method by merging multiple features from a monocular red-green-blue (RGB) image was proposed. Firstly, the High-resolution networks (HRNet) was used to extract the features of indoor corner keypoints (CKs) and informative edges (IEs) in the form of heatmaps. Secondly, the extracted features were then selected through a room type (RT) classification network. Finally, a feature fusion algorithm was proposed, which obtained the initial layout hypothesis through CK features and then continuously optimized the layout hypothesis with IE features. The proposed ILE method was validated on the large scene understanding challenge (LSUN) dataset.