Indoor scene virtual roaming method based on reflection decomposition

Abstract

The invention discloses an indoor scene virtual roaming method based on reflection decomposition, and the method comprises the steps: firstly obtaining a rough global triangular mesh model projection as an initial depth map through three-dimensional reconstruction, enabling a depth edge to be aligned to a color edge, and converting the aligned depth map into a simplified triangular mesh; detecting planes in the global triangular mesh model, and if a certain plane is a reflection plane, carrying out double-layer expression on a reflection area for each picture with the visible reflection plane to correctly render the reflection effect of the object surface; and giving a virtual view angle, drawing a virtual view angle picture by using the neighborhood picture and the triangular mesh, and drawing a reflection region by using the foreground and background picture and the foreground and background triangular mesh. According to the method provided by the invention, large-degree-of-freedom virtual roaming can be carried out in a large indoor scene with a reflection effect under the condition of a small storage requirement. The rendering effect is good, the roaming freedom degree is large, the effects of partial reflection, highlight and the like can be drawn, and the result is robust.

Description

technical field [0001] The invention relates to the technical field of image-based rendering and virtual viewpoint synthesis, in particular to a method for virtual roaming of an indoor scene based on an image-based rendering technology combined with reflection decomposition. Background technique [0002] The purpose of virtual roaming in indoor scenes is to build a system that outputs the drawn pictures of the virtual viewpoint given the internal and external parameters of the virtual camera. The existing relatively mature virtual roaming applications are mainly based on a series of panoramic pictures, and a purely rotating virtual roaming can be performed with each panoramic picture as the center. Most systems use simple interpolation to move between panoramic pictures, and the visual error is relatively large at this time . For virtual roaming with large degrees of freedom, there are many existing methods to achieve object-level observation or view point movement observat...

AI Technical Summary

Problems solved by technology

For larger indoor scenes, the latest methods can achieve relatively free viewpoint rendering, but the rendering effect is still not good enough, see Riegler and Koltun. "Free View Synthesis." Proceedings of the European Conference on Computer Vision.2020

In particular, for the various reflection types (ground, table, mirror, etc.) that exist in larger indoor scenes, there is still no system that can handle indoor roaming with such complex materials.

Images