Voice conversion method and device with emotion and rhythm

Abstract

The invention discloses a voice conversion method with emotion and rhythm, which comprises a training stage and a conversion stage, and is characterized in that a style coding layer with an attentionmechanism is used for calculating a style coding vector of a speaker; and the style coding vector and the speaker voice acoustic features are input into a self-coding network with a box link togetherfor training and conversion, and finally the acoustic features are converted into audio through a vocoder. On the basis of a traditional voice conversion method, rhythm and emotion information of a speaker is introduced, so that the converted voice has emotion and rhythm of the voice of a target speaker, and the method has high similarity and high voice quality in many-to-many speaker voice conversion tasks such as intra-set pairing, intra-set pairing, extra-set pairing, extra-set pairing and the like.

Description

technical field [0001] The invention relates to the technical field of speech processing, in particular, a method and device for converting speech with emotion and rhythm. Background technique [0002] Voice conversion is a speech technology that retains the content information of the source speaker's voice and converts it into the target speaker's voice. This technology has a wide range of application scenarios. For example, users can convert their own voices into the voices of their favorite stars, as well as the "voice-changing bow" that anime fans talk about. Synthesis, voiceprint recognition, voiceprint security and other fields are of great significance. [0003] Existing speech conversion methods have developed from parallel training data to non-parallel training data, and one-to-many transformation to many-to-many transformation. There are several ways to achieve it: one is to use a certain method to align the speech features and parameters of non-parallel corpora ...

AI Technical Summary

Problems solved by technology

[0003] Existing speech conversion methods have developed from parallel training data to non-parallel training data, and one-to-many transformation to many-to-many transformation. There are several ways to achieve it: one is to use a certain method to align the speech features and parameters of non-parallel corpora , and then train the model to obtain the speech conversion function. The corpus alignment work of this method is relatively complicated, and the speech conversion effect is relatively limited; one is to perform speech recognition on the speech data to be converted to obtain the recognized text, and then use the speech synthesis model of the target speaker to perform speech Synthesis, this method needs to rely on the development of speech recognition and personalized speech synthesis; there is another method that directly converts speech, extracting fundamental frequency features, speaker features and Content features, constructing a conversion function, but the feature extraction project of this method is more complicated, and the naturalness of the synthesized speech is low

Images