The invention discloses a method and a device for reconstructing a three-dimensional scene in real time. The method comprises a
geometric reconstruction process and a
texture reconstruction process, wherein the
geometric reconstruction process comprises the steps of inputting an
RGB image and a depth image of a scene to be reconstructed in real time, and dividing the
RGB image and the depth imageinto a
key frame and a non-
key frame for
processing according to frame similarity, and the
processing of the
key frame comprises the steps of storing feature points, performing
loopback detection to find similar frames, and performing global
pose optimization and registration; for the non-key frame, registering with a relative
pose relative to the key frame; after TSDF field fusion, grids are extracted through a moving cubic
algorithm to generate geometry of a three-dimensional scene model; wherein the
texture reconstruction process comprises the steps of restoring and realizing texture optimization through a
texture mapping mode to reconstruct color information of a three-dimensional scene model, selecting images under a plurality of visual angles from an input
RGB image sequence, extracting texture pieces, splicing the texture pieces into a complete texture, and mapping the texture pieces. According to the method, the three-dimensional model of the high-quality appearance can be reconstructed in real time.