GeoRecon: Geometric Coherence for Online 3D Scene Reconstruction From Monocular Video

Abstract

Online 3D scene reconstruction from monocular video aims to incrementally recover 3D mesh from monocular RGB videos. It enables robots to accomplish tasks involving interactions with the environment. Due to the high memory consumption of 3D data, almost all existing methods adopt the coarse-to-fine architecture, in which the voxel is progressively sparsified and split across levels. However, these methods overlook alignment between different levels, resulting in poor geometric properties of the reconstructed scene. Furthermore, the whole framework relies on voxel features for supervision, lacking effective supervision of the image geometric features extracted by the feature extraction network. These geometric features are essential for further 3D scene reconstruction. To tackle the above problems, we propose GeoRecon, which achieves geometric coherent reconstruction through keyframe 2D representation self-regression and cross-level 3D voxel feature alignment. Specifically, for 2D image space, to alleviate the lack of supervision in 2D feature extraction, an image reconstruction self-supervision regression constraint is introduced on the input 2D keyframes to ensure that the extracted features can learn accurate geometric features and further voxel features. For 3D voxel features space, to achieve consistent alignment between different levels, the high-level voxel features are used to constrain low-level voxel features, and achieve alignment from coarse (i.e., low-level) voxel features to fine (i.e., high-level) voxel features. With the design of these two components, the proposed method effectively reconstructs the geometric structures of the scene. The experimental results demonstrate the effectiveness of the proposed method.