Method and device for reconstructing three-dimensional scene in real time

Abstract

The invention discloses a method and a device for reconstructing a three-dimensional scene in real time. The method comprises a geometric reconstruction process and a texture reconstruction process, wherein the geometric reconstruction process comprises the steps of inputting an RGB image and a depth image of a scene to be reconstructed in real time, and dividing the RGB image and the depth imageinto a key frame and a non-key frame for processing according to frame similarity, and the processing of the key frame comprises the steps of storing feature points, performing loopback detection to find similar frames, and performing global pose optimization and registration; for the non-key frame, registering with a relative pose relative to the key frame; after TSDF field fusion, grids are extracted through a moving cubic algorithm to generate geometry of a three-dimensional scene model; wherein the texture reconstruction process comprises the steps of restoring and realizing texture optimization through a texture mapping mode to reconstruct color information of a three-dimensional scene model, selecting images under a plurality of visual angles from an input RGB image sequence, extracting texture pieces, splicing the texture pieces into a complete texture, and mapping the texture pieces. According to the method, the three-dimensional model of the high-quality appearance can be reconstructed in real time.

Description

technical field [0001] The invention relates to computer vision, in particular to a method and device for reconstructing a three-dimensional scene in real time. Background technique [0002] Scene reconstruction can be widely used in fields such as machine vision and automatic driving. Reconstructing realistic 3D scene models is especially important for developing interactive virtual reality (VR / AR) applications. The 3D scene reconstruction algorithm involves two aspects of geometry and texture reconstruction. In terms of geometric reconstruction, there are already many mature technical solutions to achieve large-scale high-precision and high-efficiency, while texture reconstruction technology is still limited to offline reconstruction, lacking high-efficiency and high-quality solutions. [0003] A class of real-time reconstruction technology, that is, real-time reconstruction technology based on RGBD input image and TSDF function as spatial representation, such as literat...

AI Technical Summary

Problems solved by technology

These two parts usually have relatively high complexity, so they are limited to offline reconstruction

Images