Dynamic human body free viewpoint video generation method and device based on neural network

Abstract

The invention provides a method and device for generating a dynamic human body free viewpoint video based on a neural network, and relates to the technical field of computer vision and computer graphics, and the method comprises the steps: reconstructing a pre-scanning model of a single human body; shooting the single human body through a plurality of RGB cameras to obtain an RGB image sequence; the pre-scanning model being deformed, so that the deformed pre-scanning model is matched with each frame of RGB image; sampling the pre-scanning model, defining a hidden code at each sampling point, and jointly optimizing the hidden codes and network parameters based on a neural network; and obtaining any rendering view angle, and generating a free viewpoint video based on the any rendering view angle. Therefore, the RGB image sequence is captured based on the plurality of RGB cameras, and the free viewpoint video with continuous and dynamic time domain is generated according to the sequence, so that a more real and dynamic rendering result is generated.

Description

technical field [0001] The present application relates to the technical fields of computer vision and computer graphics, and in particular to a method and device for generating a dynamic human body free-viewpoint video based on a neural network. Background technique [0002] Free-viewpoint video generation of dynamic human body is an important problem in the field of computer graphics and computer vision. High-quality human body free-viewpoint video has broad application prospects and important application value in the fields of film and television entertainment and human body digitization. However, high-quality, high-fidelity human body free-viewpoint video acquisition usually relies on expensive laser scanners or multi-camera array systems to model the human body. Although the effect is more realistic, there are also some obvious shortcomings: first, The equipment is complicated, and these methods often require the construction of multi-camera arrays; second, the speed is...

AI Technical Summary

Problems solved by technology

However, high-quality, high-fidelity human body free-viewpoint video acquisition usually relies on expensive laser scanners or multi-camera array systems to model the human body. Although the effect is more realistic, there are also some obvious shortcomings: first, The equipment is complicated, and these methods often require the construction of multi-camera arrays; second, the speed is slow, and it often takes at least 10 minutes to several hours to reconstruct a 3D human body model, and the reconstruction sequence is even slower

Images