New viewpoint image synthesis method based on depth-aided full resolution network

Abstract

The invention provides a new viewpoint image synthesis method based on a depth-aided full resolution network. The main content comprises the depth-aided full resolution network, the loss function andtraining. The process comprises the steps that the encoder part of the full resolution network firstly extracts important local features from the input image; then a depth predictor performs pre-training on the large image data set after detecting the global image information so as to estimate the depth map of the input image; then the local features and the depth are fed to the decoder and dual channel mapping of the target viewpoint position is indicated; and finally the decoder converts the combined features into the distortion field so as to synthesize the final target image based on flowdeformation. The full resolution network is designed and the local image features are extracted according to the same input resolution so that the blurred ghost in the finally synthesized image can beprevented, and the high-resolution and high-quality image can be acquired.

Description

technical field [0001] The present invention relates to the field of image synthesis, in particular to a new viewpoint image synthesis method based on a depth-assisted full-resolution network. Background technique [0002] New viewpoint image synthesis is an interdisciplinary field of computer vision and image processing. It is an important part of virtual reality technology and has a wide range of applications in many aspects. For example, new viewpoint synthesis based on face images is an important application field in face processing, widely used in face recognition, face animation and many other aspects; by inputting existing face images, using the same viewpoint for the input image Samples can be expressed, and by combining and synthesizing new viewpoint images, enough images from different angles of the face can be obtained, which will help provide more effective information for the detection of criminal cases. In the research and development of digital TV in the futu...

AI Technical Summary

Problems solved by technology

[0004] Aiming at the problem that global feature prediction does not change local details but cannot modify local details, the purpose of the present invention is to provide a new viewpoint image synthesis method based on depth-assisted full-resolution network. The encoder part of the full-resolution network first Important local features are extracted from the input image, and then the depth predictor is pre-trained on a large image dataset by detecting the global image information to estimate the depth map of the input image, and then feeds the local features and depth to the decoder and instructs Two-channel mapping of the target viewpoint position, and finally flow-based deformation, where the decoder converts the combined features into a warp field to synthesize the final target image

Images