Cross-language timbre conversion system and method based on zero-order learning

Abstract

The invention discloses a cross-language timbre conversion system and method based on zero-order learning. The system sequentially comprises a mixed phoneme recognition module, a timbre conversion module, a speaker coding module and a vocoder module. According to the system, a voice signal Mel spectrum serves as an input signal, bottleneck features of the voice signal Mel spectrum are extracted through the phoneme recognition module, the features are normalized and then transmitted to an acoustic model, the Mel spectrum synthesized by the acoustic model is controlled by controlling a speaker reference vector, and finally audio is synthesized through a vocoder. The system can convert the voice of a common speaker into the timbre of a specified speaker, is suitable for accent corpora which do not appear in a training database, can be suitable for voice change of dialects in multiple regions, and has a wide application prospect.

Description

technical field [0001] The invention belongs to the field of speech synthesis technology, in particular to a zero-shot learning-based cross-language timbre conversion system and method. Background technique [0002] Voice Conversion (Voice Conversion) technology is designed to convert the timbre of a voice into another specified person's timbre (B.Sisman, J.Yamagishi, S.King and H.Li,"An Overview of VoiceConversion and Its Challenges:From Statistical Modeling to Deep Learning," in IEEE / ACM Transactions on Audio, Speech, and Language Processing, vol.29, pp.132-157, 2021, doi:10.1109 / TASLP.2020.3038524.). It is a branch of the field of speech synthesis, but it is not only speech synthesis, but also involves technologies in speech recognition and speaker recognition related fields. One of the core issues of timbre conversion technology is how to decouple the content and timbre of the speech, which not only ensures the integrity of the speech content, but also ensures that the ...

AI Technical Summary

Problems solved by technology

[0005] The problem to be solved by the present invention is to provide a cross-language timbre conversion method based on zero-time learning, which solves the common problems of timbre conversion models such as GAN and VAE from three aspects. The extracted features are heavily dependent on the data distribution of the training set, resulting in poor generalization ability of the trained model. Second, the benchmark method does not fully decouple semantics and timbre, and does not explicitly separate the speech content from the speaker's timbre. , the third is that it is difficult for the benchmark method to retain the pronunciation habits of a specific language for cross-lingual and dialect-accented speech, and it performs poorly in language transfer

The present invention can extract high-level semantic features with the aid of a phoneme recognition model that highly strips speaker information, and convert it into the target speaker's voice in combination with the target speaker feature vector and fundamental frequency, and solves the problem that does not exist in the training data by using the phoneme feature as a model. Questions related to the language, so that the model can convert a variety of dialect accent speech, foreign language speech

Images