Video flow based three-dimensional dynamic human face expression model construction method

Abstract

The invention relates to a three-dimension dynamic face pathetic model construction method based on video flow, which can return the three-dimension face pathetic based on input video flow, wherein the algorism comprises: (1) marking face character point at the first frame of input video; (2) using light stream method of affine correction to track the character point; (3) rebuilding the two-dimension track data based on factor decomposition into three-dimension data; (4 using rebuilt three-dimension date to match general face model, to generate personal face and dynamic pathetic motion; (5) using character face technique to compress the original video; (6) using character face to rebuild input video and projecting dynamic pattern, to compose true virtual appearance. The invention has high time / spatial efficiency and high value.

Description

technical field [0001] The invention relates to the intersecting fields of computer vision and computer graphics, in particular to a three-dimensional dynamic facial expression modeling method based on video streams. Background technique [0002] Personalized face modeling and realistic expression animation generation have always been a challenging topic, and have been widely used in virtual reality, film production, game entertainment, etc. Since the pioneering work of Parke [1] in 1972, research on face and expression modeling has made great progress. According to the different input data required, the modeling methods are mainly divided into the following categories: modeling based on captured 3D sample data; modeling based on images; modeling based on video streams. Blanz et al. [2] build a personalized face model based on an input face image by learning the statistical features in the 3D face database, which requires the use of expensive laser scanning equipment to pre...

AI Technical Summary

Problems solved by technology

Literature [5] uses normalized orthogonal images to model faces and uses muscle vectors to drive expressions. The disadvantage is that the position of muscle vectors is difficult to set correctly, and the orthogonal constraints are too strict, which makes the method lack of generalization

Literature [6] uses two frontal images to model the face. The camera must be calibrated in advance, and the reconstructed feature points are relatively small. It is difficult to accurately reflect the local features of the face only by interpolating the feature points to generate a face grid.

Literature [7] also uses orthogonal images to optimize the face model through a step-by-step refinement process, which also has the disadvantage of too strict constraints

Li Zhang et al. [8] used structured light to reconstruct human facial expressions from video streams through stereo vision, which requires hardware devices including structured light projectors, and the scanned model needs cumbersome manual preprocessing. Ambient light requirements are high

The method proposed by Zicheng Liu et al. [9] is very meaningful, that is, reconstructing a 3D face model from an uncalibrated video stream. This method has no strict requirements on the input data, but the detection and matching of corner points is not robust enough and is vulnerable to The influence of lighting, which may cause the failure of reconstruction work

[0003] The traditional face animation method mainly considers the geometric deformation of the face model [5, 6, 7, 9]. The texture is mapped to the vertices of the model, so when the mesh deforms, the texture will also stretch and distort accordingly. Therefore, the traditional Texture mapping can be regarded as a static method. However, the human face is a highly non-rigid surface, and facial expressions include not only small geometric deformations (such as wrinkles) on the surface, but also changes in skin color and expression, and purely from the perspective of geometric deformation It is difficult to simulate these changes. Therefore, in this sense, traditional texture mapping methods are not enough to produce highly realistic facial expressions.

Images