Direct speech-to-speech translation via machine learning

Abstract

The present disclosure provides systems and methods that train and use machine-learned models such as, for example, sequence-to-sequence models, to perform direct and text-free speech-to-speech translation. In particular, aspects of the present disclosure provide an attention-based sequence-to-sequence neural network which can directly translate speech from one language into speech in another language, without relying on an intermediate text representation.

Description

[0001] related application [0002] This application claims priority and benefit to US Provisional Patent Application No. 62 / 826,258, which is hereby incorporated by reference in its entirety. technical field [0003] This disclosure relates generally to machine learning. More specifically, the present disclosure relates to direct and text-free speech-to-speech translation by machine learning models, such as sequence-to-sequence models. Background technique [0004] Speech-to-speech translation (S2ST) refers to the process of translating speech in one language (eg, represented by a first speech waveform) into speech in a different language (eg, represented by a different second speech waveform). Conventional S2ST systems rely on a cascaded approach that combines multiple disparate systems to perform translation. In particular, conventional S2ST systems are often divided into three components that operate separately and sequentially: automatic speech recognition (ASR), text...

AI Technical Summary

Problems solved by technology

Finally, cascaded systems cannot learn to produce fluent pronunciations of words that do not require translation, such as names or other proper nouns

Images