Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

A method for 3D reconstruction and texture generation from single-view face based on multi-task learning

A multi-task learning and 3D reconstruction technology, which is applied in the field of single-view face 3D reconstruction and texture generation based on multi-task learning, can solve problems such as inconvenience, texture errors and limitations in occluded areas, and achieve convenient post-processing and data generation Comprehensive, Fast Effects

Active Publication Date: 2019-01-22
NANJING UNIV
View PDF8 Cites 55 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The texture image restored by this method is better, but the reconstruction effect of the 3D model of the face is not good enough.
[0005] The above-mentioned existing technologies all have the following disadvantages: based on a single face image, the accurate three-dimensional geometric structure of the face and the complete texture cannot be restored at the same time, and for the method of deep learning, the collection cost of the complete texture expansion map data set is very high and inconvenient
Some methods can obtain a relatively complete three-dimensional geometric structure of the face, but the texture part is a texture image obtained directly from the local affine transformation of the input original image, and the texture in the occluded area is obviously wrong
Some methods can restore high-resolution and complete face texture maps, but they are limited to frontal face images or small-angle side face images, and the restoration of the three-dimensional structure of the face is not ideal.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A method for 3D reconstruction and texture generation from single-view face based on multi-task learning
  • A method for 3D reconstruction and texture generation from single-view face based on multi-task learning
  • A method for 3D reconstruction and texture generation from single-view face based on multi-task learning

Examples

Experimental program
Comparison scheme
Effect test

Embodiment

[0072] This embodiment provides a method for single-view face 3D reconstruction and texture generation based on multi-task learning, specifically including:

[0073] (1) In order to use the 300W_LP face database to make a data set, first restore the grid structure of the 3D model of the face from the geometric shape parameters in the 3DMM model parameters corresponding to the face images in the natural scene, and then use the 3DMM model parameters The texture parameters in the method and the additional light source parameters restore the texture information of the 3D model of the face. The 300W_LP face database includes four sub-datasets of AFW, HELEN, IBUG, and LFPW, which contain 3837 faces of different identities. At the same time, the faces of each identity have images from different angles, including images from the left side to the middle of the right side Insert images with a number of viewpoints ranging from 9 to 17. They all correspond to the 3D model of the face of ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a method for 3D reconstruction and texture generation from single-view face based on multi-task learning, belonging to the computer vision field. The method comprises the following steps of: selecting a special viewpoint for rendering a three-dimensional human face model; generating depth map and texture map as truth value data from special viewpoint; designing an integrated learning and coding network based on feature sharing of depth information and texture information; designing the branch decoding network to recover the depth map from the shared features, and recovering the depth map, designing the mutual information with shared features as latent variables to maximize the generation of antagonistic network, and restoring the texture unwrapping map, adjusting the proportion of each task loss function and training the model; interpolating the depth map and using texture map to reconstruct the 3D face mesh model with texture details. The invention utilizes multi-task learning to carry out single-view face three-dimensional reconstruction, texture generation and style migration, and has the advantages of high speed, low cost and the like.

Description

technical field [0001] The invention relates to the field of computer vision, in particular to a single-view human face three-dimensional reconstruction and texture generation method based on multi-task learning. Background technique [0002] Three-dimensional face models are widely used in security authentication, film and television animation, medical science and other fields. However, it is very expensive to obtain the accurate 3D structure of the face and the complete high-resolution texture map at the same time, and the obtained texture map is not convenient for post-processing, or there are various problems such as that the 3D structure and the high-resolution texture map cannot be obtained at the same time. problem. [0003] For single-view face 3D reconstruction and texture generation using traditional methods, there are usually two techniques: (1) Shape-from-Shading (SFS) method or photometric stereo (Photometric stereo) method is used according to the single-view ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06T17/00G06T7/50G06T7/80
CPCG06T7/50G06T7/80G06T17/00G06T2207/30201
Inventor 曹汛汪晏如朱昊张艺迪
Owner NANJING UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products