Voice-based mouth shape animation synthesis device and method and readable storage medium

Abstract

The invention discloses a voice-based mouth shape animation synthesis device. The device comprises a memory and a processor, wherein a mouth shape animation synthesis program capable of running on theprocessor is stored on the memory, and the program is executed by the processor through the steps that target text data is acquired, and phonemic characteristics in the target text data are acquiredaccording to a pronunciation dictionary; the phonemic characteristics are input into a pre-trained deep neural network model to output acoustic characteristics, and the acoustic characteristics are input into a voice synthesizer to output voice data; according to the voice data, a pre-trained tensor model and speaker identification information, mouth shape data is acquired; and a mouth shape animation corresponding to the voice data is generated according to the mouth shape data. The invention furthermore provides a voice-based mouth shape animation synthesis method and a computer readable storage medium. Through the voice-based mouth shape animation synthesis device and method and the computer readable storage medium, the technical problem that a mouth shape animation which is matched with synthesized voice data and has a sense of reality cannot be displayed in the prior art is solved.

Description

technical field [0001] The invention relates to the field of computer technology, in particular to a voice-based lip animation synthesis device, method and readable storage medium. Background technique [0002] Speech synthesis, also known as text-to-speech technology, is a technology that can convert text information into speech and read it aloud. It involves multiple disciplines such as acoustics, linguistics, digital signal processing, and computer science. It is a cutting-edge technology in the field of Chinese information processing. The main problem to be solved is how to convert text information into audible sound information. [0003] In some application scenarios, such as computer-aided pronunciation training application scenarios, it is necessary to dynamically display the mouth shape changes of the speaker when playing voice data to help users perform pronunciation training. In the prior art, synthetic When there is no real speaker's mouth shape data correspondin...

AI Technical Summary

Problems solved by technology

[0004] The present invention provides a voice-based lip animation synthesis device, method and readable storage medium, the main purpose of which is to solve the technology in the prior art that cannot display realistic lip animations that match the synthesized voice data question

Images