Viewpoint data generation method and device based on convolutional neural network

Abstract

The invention provides a viewpoint data generation method and device based on a convolutional neural network, and the method comprises the steps: obtaining discrete viewpoint data sets of a pluralityof training scenes; extracting first viewpoint data of a first viewpoint and second viewpoint data of a second viewpoint in multiple frames of viewpoint data for the same training scene, and obtaininga first camera parameter and a second camera parameter; inputting the first color image information, the second color image information, the first camera parameter and the second camera parameter into a preset convolutional neural network; estimating viewpoint data through a convolutional neural network; and calculating a loss value between the third viewpoint data and the estimated viewpoint data, and completing training of the convolutional neural network so as to facilitate generation of a viewpoint image. According to the method, new viewpoints generated by joint optimization in combination with depth information can have more stable information, ghosting and black holes generated by large parallax change are reduced, and continuous and consistent dense viewpoint generation under thecondition of sparse viewpoint input is realized.

Description

technical field [0001] The present invention relates to the technical field of computer vision, in particular to a method and device for generating viewpoint data based on a convolutional neural network. Background technique [0002] Image-based new viewpoint rendering is an important issue in the field of 3D reconstruction and computer vision. With the development and popularization of virtual reality technology and augmented reality technology, the problem of generating continuous and dense viewpoints from sparse viewpoint images to achieve visual continuity and immersion has attracted more and more attention. When some specific application scenarios can only obtain limited images, such as Google Street View can only obtain discrete scene images taken by users. In order to reconstruct a continuous immersive street view, the algorithm for generating dense viewpoint images from sparse viewpoints becomes particularly important. important. [0003] In the existing technology...

AI Technical Summary

Problems solved by technology

[0003] In the existing technology, the new viewpoint rendered by viewpoint interpolation or depth map has the defects of ghosting and hollowness, so how to solve this practical problem has become a challenging issue

Images