Method for semantic completion of single depth map point cloud scene

Abstract

The invention provides a method for semantic completion of a single depth map point cloud scene, and belongs to the field of three-dimensional reconstruction in the field of computer vision. Accordingto the method, viewpoint repairing loopholes are converted in the process of mutual projection of a depth map, a depth segmentation map and point cloud, and high-resolution point cloud completion andsemantic segmentation are carried out at the same time. The problems that voxel representation form resolution is low and a point cloud representation form cannot give consideration to semantic segmentation in the scene semantic completion problem are solved, and a high-resolution geometric structure and semantic information details of the scene can be replenished at the same time by performing scene semantic completion on the three-dimensional point cloud; based on a single depth map, tasks of three-dimensional point cloud completion and semantic segmentation can be completed at the same time; effectiveness of semantic information and three-dimensional geometrical information constraints on semantic completion of the point cloud scene is verified.

Description

technical field [0001] The invention belongs to the field of three-dimensional reconstruction (3D Reconstruction) in the field of computer vision. The specific realization result is point cloud semantic completion of indoor scenes, and in particular relates to a method for simultaneously performing surface completion and semantic segmentation. Background technique [0002] The task of semantic scene reconstruction in 3D reconstruction is the process of recovering 3D scenes from 2D images and obtaining semantic information. With the help of depth information, 3D scene reconstruction can be more accurate and reliable. However, the collected depth map is often incomplete due to occlusion and fixed viewpoint, so it is very important to understand and reconstruct the local depth map. From the early voxel method, to the end-to-end deep convolutional neural network architecture, to the fusion of RGB and depth information, the method of semantic scene completion has been improving ...

AI Technical Summary

Problems solved by technology

However, these methods all use point clouds as input, directly process 3D points through different networks, and cannot achieve the effect of generating a complete point cloud with semantic labels from a local depth map.

Images