Three-dimensional head model reconstruction method and device based on deformable nerve radiation field

Abstract

The invention discloses a three-dimensional head model reconstruction method and device based on a deformable neural radiation field, and the method comprises the steps: carrying out the segmentation and face key point detection of an input video frame by frame, and fitting a parameterized model frame by frame; semantic information is extracted from an input video frame by frame, wherein the semantic information mainly comprises a hair label and a face label; under the guidance of a parameterization model and semantic information, rigid registration coding, non-rigid deformation coding, texture coding, a deformation quantity estimation model, a topology estimation model, a signed distance field estimation model, a color estimation model and a semantic information estimation model are optimized on an input video in a deformable neural radiation field rendering mode; therefore, a frame-by-frame high-quality three-dimensional head model is obtained.

Description

technical field [0001] The invention relates to the technical field of image processing, in particular to a method and device for reconstructing a three-dimensional head model based on a deformable nerve radiation field. Background technique [0002] In recent years, with the development of computer technology and communication technology, digital human representation based on artificial intelligence has attracted more and more attention, among which realistic head representation plays a very important role in the credibility of virtual digital human. Furthermore, as the most representative part of a person's body, the human head can provide a wealth of information, such as: a person's gender, age, emotion, race, health status, personality, and so on. High-quality 3D head reconstruction has always been a research hotspot in the fields of computer vision and graphics, and it has a wide range of applications in scenes such as human body digitization and animation. Therefore, ...

AI Technical Summary

Problems solved by technology

[0003] In the past, there were mainly four channels for obtaining high-quality 3D head models: Human body scanning, which collects data by scanning real people with the help of scanning equipment. This method can directly obtain 3D information of people. The required scanning equipment is expensive, and it is generally impossible to model the hair. The scanned data is also accompanied by deletion and noise, and subsequent data processing is still required; based on the reconstruction of multiple images, it is collected from multiple perspectives or under multiple lights 3D reconstruction of images, this method still requires the construction of specific acquisition equipment, or the setting of lighting conditions, it still has a certain degree of difficulty and cost in data acquisition, and this method usually requires a complex optimization process to Guaranteed geometric accuracy; based on the reconstruction of a single image, 3D reconstruction is performed from the collected single image. This method greatly simplifies the process of data acquisition and makes data acquisition convenient and fast. The information is too single, so the 3D geometry reconstructed from a single image may contain ambiguity, which is still not conducive to put into use; based on video sequence reconstruction, the corresponding 3D head model is reconstructed from a video of a person, and then With the acquisition of portable electronic devices, the acquisition of a piece of video is becoming faster and faster. This method usually uses a parametric model of the face or head to perform 3D reconstruction by fitting each frame. The advantage of this is that it can Good geometric structure information is obtained, but due to the limitation of the expressive ability of the parametric model, the geometric accuracy of this type of method is low, and the hair cannot be represented

Images