Face motion synthesis method based on voice driving, electronic equipment and storage medium

Abstract

The embodiment of the invention relates to the technical field of computer information, and discloses a face motion synthesis method based on voice driving, electronic equipment and a storage medium. The method comprises the following steps: processing a voice signal of a to-be-recognized face motion to obtain an audio vector corresponding to the voice signal; inputting the audio vector into a parameter recognition model for processing, and outputting a face muscle motion parameter corresponding to the to-be-recognized face motion; and moving angular points on a plurality of elastomers divided according to face muscle distribution in a face model through the face muscle motion parameters of the to-be-recognized face motion, and obtaining a to-be-recognized face motion result. The scheme can be universally applied to a character model containing a plurality of angular point numbers, and the output human face motions are rich and the expression effect is natural.

Description

technical field [0001] Embodiments of the present invention relate to the field of computer information technology, and in particular to a method for synthesizing human facial movements based on voice drive, electronic equipment and storage media. Background technique [0002] Whether it is a robot in reality or a virtual character or model in a computer, how to automatically lip-sync a virtual character or model through audio is a difficult problem in the industry. Even after years of research and development, this problem still plagues relevant practitioners personnel. [0003] At present, there are many ways to drive the mouth shape of virtual characters based on voice, the most commonly used is the VOCA (Voice Operated Character Animation) model. The target data for VOCA model training is the corner position of the character model virtualized by 3D visual effects synthesis software such as FALME. However, since the number of corner points of the character model synthesi...

AI Technical Summary

Problems solved by technology

The target data for the training of the VOCA model is the corner position of the character model virtualized by 3D visual effects synthesis software such as FALME. Since the number of corner points of the character model synthesized by FLAME is fixed, it is difficult to migrate the target data to a custom one. In virtual characters, it is impossible to achieve the effect of one-time training and multi-scenario application

In addition, the voca model usually only models mouth movements, and there are no movements in many other parts of the face, such as raising eyebrows, blinking, etc., which will cause the output face movement effect to be stiff

Images