Real-time facial expression reconstruction method

Abstract

The invention discloses a real-time facial expression reconstruction method. The method comprises the following steps of artificially manufacturing 29 facial form models and 46 expression models; carrying out face key point detection on one or more input pictures or videos; utilizing key points obtained by input photo or video detection and key points of a pre-calibrated human face three-dimensional model to perform face shape fitting on the human face key points through an error minimization criterion, and extracting human face textures through photos or videos to reconstruct an expression-free human face three-dimensional model. For a subsequently input video, face key points in the video are detected in real time, and face expression parameters are fitted in a mode of minimizing errorsbetween the key points and model key points, and the facial expression parameters are applied to the reconstructed expression face model to realize real-time reconstruction of the facial expression. According to the method, the facial expression of a single person can be reconstructed, and the facial expression of one person can also be applied to the reconstruction of another face, so that real-time face replacement is realized.

Description

technical field [0001] The invention relates to the technical field of three-dimensional reconstruction, in particular to a method for real-time reconstruction of three-dimensional facial expressions with a sense of reality. Background technique [0002] 3D face reconstruction is widely used in face recognition, 3D film and television animation production and other fields. The most classic method in 3D face reconstruction comes from the 3D Morphable Model (MorphableModel) proposed by Thaomas Vetter et al. in 1999. That is, a new face is obtained through the linear combination of n faces. The principle is as follows: [0003] S model = S avr +S i ·α i T model =T avr +T i ·β i [0004] Among them, S model is the fitted face model, S avr is the average face model, S i is the deviation between the i-th face and the average face. T model is the fitted face texture, T i is the difference between the i-th texture and the average texture. The essence of the method...

AI Technical Summary

Problems solved by technology

However, this method based on optical fluid flow measurement is susceptible to nonrigid motion and lighting conditions, and if the storage quality of the picture is not good or the smoothness of the video is not good, the accuracy of expression reconstruction will be affected.

Images