RGB-D image-based indoor scene three-dimensional reconstruction method

Abstract

The invention discloses a method based on RGB-. According to the indoor scene three-dimensional reconstruction method of the D image, a semantic segmentation result is used for repairing a depth imagecavity, object contour and category information is provided for three-dimensional reconstruction, and the shape and appearance of an object are obtained according to priori knowledge, so that more accurate data is provided for three-dimensional reconstruction. Three-dimensional reconstruction provides three-dimensional space information for semantic segmentation, and the problem of false segmentation caused by object overlapping, illumination influence and the like in two-dimensional image segmentation is solved. Multi-level camera pose estimation and sparse feature matching are used for providing a rough estimation pose, then a precise camera pose is obtained through a dense geometric and photometric optimization method, and a more precise camera pose is provided for model reconstruction. In the reconstruction process, each frame is locally optimized, meanwhile, a key frame mechanism is added, global optimization and closed-loop detection are established, constraint is established for spatial points corresponding to pixels of the key frames, error accumulation is effectively inhibited, the pose of the camera is further optimized, and the precision of a reconstruction result is improved.

Description

technical field [0001] The invention belongs to the technical field of computer vision, and more specifically relates to a method for three-dimensional reconstruction of indoor scenes based on RGB-D images. Background technique [0002] The principle of the depth camera Kinect is that the infrared emitter emits infrared rays and irradiates the surface of the object to form random reflection speckles, which are then received by the depth sensor, and then the system chip operates to generate a depth image. For transparent materials and planes with missing textures, infrared rays cannot be reflected to form speckle or the effect is poor, resulting in a depth image with holes. At present, most research works use bilateral filtering method to preprocess the depth image simply. [0003] In the prior art, the 3D reconstruction based on RGB-D images mainly includes: Newcombe et al. directly calculate the 3D coordinates of the spatial points through the preprocessed depth image, and...

AI Technical Summary

Problems solved by technology

[0005] Aiming at the defects of the prior art, the purpose of the present invention is to solve the technical problems of the prior art that the reconstruction accuracy is not high and cannot solve the mis-segmentation caused by the hole in the depth image

Images